Clustered Architecture Patterns:
  Delivering Scalability and Availability
                                               ...
Agenda

                    Patterns from Years of Tier-Based Computing

                    Network Attached Memory / J...
The Innocent Enterprise Application

                    Within every innocent enterprise application lies a sleeping
   ...
The State Monster




Confidential – for information of designated recipient only. Copyright Terracotta 2006
The State Monster

                                                                                           User-scoped...
The State Monster

                    At Walmart.com we started like everyone else: stateless + load-balanced
          ...
The Precarious Path: Our tools lead us astray

                    Stateless load-balanced architecture ⇒ bottleneck on D...
CRUD Piles Up…

                    Types of clustering:
                                –  Load-balanced (non-partitione...
The Stateless Convention: Use The Database




Confidential – for information of designated recipient only. Copyright Terr...
The Stateless Convention: Push State to the Database




                                                                 ...
The Stateless Convention: Push State to Peers




Confidential – for information of designated recipient only. Copyright T...
The Stateless Convention: Push State to Peers




             Push state everywhere gives HA, but doesn't scale

       ...
The Stateless Convention: Push State to Client

                    Highly scalable for conversation state UNLESS state i...
The Stateless Convention: Eats Developer Brains




Confidential – for information of designated recipient only. Copyright...
The Stateless Convention: Eats Developer Brains

                                                                         ...
Changing the Assumptions: JVM-level Clustering




Confidential – for information of designated recipient only. Copyright ...
What Terracotta Is


  Definition

                    From “The Definitive Guide…”
                                –  Te...
What Terracotta Is


  Terracotta Deployment Model




                                    Your JVMs
                     ...
What Terracotta Is


  Changes In 1 JVM Visible in Others




                                 JVM                        ...
What Terracotta Is


  …Like This




                                 JVM                                                ...
What Terracotta Is


  Oh Yeah: Did I Mention It is JVM-Level?


                                 Before Terracotta       ...
Performance + Reliability + Simplicity

                    10X throughput over conventional APIs
                       ...
What Terracotta Is


  Batching and Ordering In Accordance With JMM

                    Durable

                    Tr...
What Terracotta Is


  What TC Does for You

                    Pure Java
                                –  Shares your...
What Terracotta Is


  Reduces Clustering to Tuning

                    Always coherent

                    Always on ...
When to use Terracotta: Will It Work With My App?
                                                                        ...
When to Use Terracotta


  Users and Use Cases

                    Web Apps                                             ...
When to Use Terracotta


  Terracotta is a Platform (not an app server)

                    How do you share objects
   ...
When to Use Terracotta


  API: Using Patterns That Are Clusterable

                    Conversation clustering / persis...
When to Use Terracotta


  Pattern: Conversation Clustering


                                                            ...
Account Creation

                    SignupController
                                –  UserValidator
                 ...
Account Creation: RegistrationService

                  DefaultRegistrationService
                  public class Default...
Account Creation

                    Durable Java data structure as temporary storage for confirmation
                 ...
AuthC/AuthZ

                    Spring Security TIM
                                –  Authentication (AbstractAuthentic...
AuthC/AuthZ: Spring Security




Confidential – for information of designated recipient only. Copyright Terracotta 2006
AuthC/AuthZ: LoginController
           @Controller	
           @RequestMapping(quot;/login.doquot;)	
           public cl...
Conversation State: Take The Exam

                    Spring WebFlow
                                –  Workflow through...
Conversational State: List Available Exams

                      <view-state id=quot;chooseExamquot; view=quot;exam/listq...
this.cacheManager = CacheManager.getInstance();	
                                                                         ...
Conversation State: Start Exam
           <action-state id=quot;startExamquot;>	
             <on-entry>	
               <...
Conversation State: Start Exam (cont.)

    public class ExamSessionServiceImpl implements ExamSessionService {	
      // ...
Coherent View Of Global Data: View Ongoing Exams
      @Controller	
      @RequestMapping(quot;/exam/ongoing.doquot;)	
   ...
Coherent View Of Global Data: View Ongoing Exams

    public class ExamSessionServiceImpl implements ExamSessionService {	...
Conversation State And Coherent View Of Global Data

                    Conversation State
                             ...
Case Study: Conversation Clustering




Confidential – for information of designated recipient only. Copyright Terracotta ...
Large Publisher Gets Caught Down the Path with Oracle




                                                                ...
Breaking the Pattern without Leaving “Load-Balanced” World
                • $1.5 Million DB & HW savings
                ...
Results

                    Database was still the SoR which kept reporting and backup
                     simple

    ...
When to Use Terracotta


  Pattern: Window Onto Large Dataset

                    Isolate the JVMs from each other caref...
When to Use Terracotta


  Pattern: Data As A Service (impl coming December)

                    Service Oriented

     ...
When to Use Terracotta


  Pattern: Cluster Membership

                    Want to know when L1 JVMs come and go
       ...
When to Use Terracotta


  Pattern: Change Notification (impl coming soon)

                    Use this one for write-be...
When to Use Terracotta


  Pattern: Shared Metadata

                    Composite Pattern…



                    Use E...
Summary
                    Simplicity, scalability, and availability can be friends

                    Write normal J...
Resources


                      Open Source (MPL-based) JVM-level clustering:
                       http://www.terraco...
Upcoming SlideShare
Loading in …5
×

NoVa Jug Sept. 08 Terracotta Clustered Architecture Patterns

2,356 views
2,305 views

Published on

Sept. 2008 presentation by Orion Letizi of Terracotta to the Northern Virginia Java Users' Group

Published in: Technology, Education
0 Comments
5 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
2,356
On SlideShare
0
From Embeds
0
Number of Embeds
10
Actions
Shares
0
Downloads
0
Comments
0
Likes
5
Embeds 0
No embeds

No notes for slide

NoVa Jug Sept. 08 Terracotta Clustered Architecture Patterns

  1. 1. Clustered Architecture Patterns: Delivering Scalability and Availability Orion Letizi – Terracotta Co-Founder and Software Engineer Confidential – for information of designated recipient only. Copyright Terracotta 2006
  2. 2. Agenda   Patterns from Years of Tier-Based Computing   Network Attached Memory / JVM-level clustering   Clustered Architecture Patterns   In-depth Review Of The New “Examinator” Reference Implementation Confidential – for information of designated recipient only. Copyright Terracotta 2006
  3. 3. The Innocent Enterprise Application   Within every innocent enterprise application lies a sleeping monster   There comes a time when every successful enterprise application outgrows its single machine architecture   Whether for high-availability or scalability or both, the adult enterprise application must live on more than one application server   That's when the latent beast strikes... Confidential – for information of designated recipient only. Copyright Terracotta 2006
  4. 4. The State Monster Confidential – for information of designated recipient only. Copyright Terracotta 2006
  5. 5. The State Monster   User-scoped session state –  Conversational state across requests   Application state –  Global data, caches, indexes, etc. Confidential – for information of designated recipient only. Copyright Terracotta 2006
  6. 6. The State Monster   At Walmart.com we started like everyone else: stateless + load-balanced + Oracle (24 cpus, 24GB)   Grew up through distributed caching + partitioning + write behind   We realized that “ilities” conflict –  Scalability: avoid bottlenecks –  Availability: write to disk ( and I/O bottleneck ) –  Simplicity: No copy-on-read / copy-on-write semantics (relentless tuning, bug fixing)   And yet we needed a stateless runtime for safe operation –  Start / stop any node regardless of workload –  Cluster-wide reboot needed to be quick; could not wait for caches to warm   The “ilities” clearly get affected by architecture direction and the stateless model leads us down a precarious path Confidential – for information of designated recipient only. Copyright Terracotta 2006
  7. 7. The Precarious Path: Our tools lead us astray   Stateless load-balanced architecture ⇒ bottleneck on DB   In-memory session replication ⇒ bottleneck on CPU, Memory   Clustered DB cache ⇒ bottleneck on Memory, DB   Memcache ⇒ bottleneck on server   JMS-based replicated cache ⇒ bottleneck on network   …Pushing the problem between our app tier CPU and the data tier I/O Confidential – for information of designated recipient only. Copyright Terracotta 2006
  8. 8. CRUD Piles Up…   Types of clustering: –  Load-balanced (non-partitioned) Scale Out –  Partitioned Scale Out   Both Trade-off Scalability or availability (usually by hand) in different ways scalability availability   …and everything we do forces the trade-offs Confidential – for information of designated recipient only. Copyright Terracotta 2006
  9. 9. The Stateless Convention: Use The Database Confidential – for information of designated recipient only. Copyright Terracotta 2006
  10. 10. The Stateless Convention: Push State to the Database   DB becomes a bottleneck   Not optimized for object data Confidential – for information of designated recipient only. Copyright Terracotta 2006
  11. 11. The Stateless Convention: Push State to Peers Confidential – for information of designated recipient only. Copyright Terracotta 2006
  12. 12. The Stateless Convention: Push State to Peers   Push state everywhere gives HA, but doesn't scale   Push conversational state to buddy can't survive buddy pair death or full cluster restart. Doesn't help for application state. Confidential – for information of designated recipient only. Copyright Terracotta 2006
  13. 13. The Stateless Convention: Push State to Client   Highly scalable for conversation state UNLESS state is large   Doesn't help with application state Confidential – for information of designated recipient only. Copyright Terracotta 2006
  14. 14. The Stateless Convention: Eats Developer Brains Confidential – for information of designated recipient only. Copyright Terracotta 2006
  15. 15. The Stateless Convention: Eats Developer Brains   Pushing state out of app server doesn't get rid of state. There is no such thing as quot;statelessquot;   Serialization/externalization: –  Full-object graph serialization –  breaks object identity –  corrupts data and programming model   quot;with a little ingenuity...quot;-- seems fun/easy to try, but ends up taking over the development effort Confidential – for information of designated recipient only. Copyright Terracotta 2006
  16. 16. Changing the Assumptions: JVM-level Clustering Confidential – for information of designated recipient only. Copyright Terracotta 2006
  17. 17. What Terracotta Is Definition   From “The Definitive Guide…” –  Terracotta is Java infrastructure software that allows you to scale your application for use on as many computers as needed, without expensive custom code or databases   NAS but for memory (almost) –  A server process for central storage and locking –  A transparent client I/O driver –  A network protocol   Network Attached Memory (NAM) Confidential – for information of designated recipient only. Copyright Terracotta 2006
  18. 18. What Terracotta Is Terracotta Deployment Model Your JVMs 1.  Uses Hotspot JVM 2.  TC libraries are jars and bootclassloader 3.  Terracotta Server is pure Java TC Libraries 4.  Terracotta Stores all data to disk 5.  Clustering in the box TCP/IP Terracotta Terracotta Server Server JVM JVM replication disk disk Confidential – for information of designated recipient only. Copyright Terracotta 2006
  19. 19. What Terracotta Is Changes In 1 JVM Visible in Others JVM JVM JVM Terracotta Confidential – for information of designated recipient only. Copyright Terracotta 2006
  20. 20. What Terracotta Is …Like This JVM JVM JVM mutate ACK ACK Establish single update Persistent TCP broadcast Connection Terracotta Confidential – for information of designated recipient only. Copyright Terracotta 2006
  21. 21. What Terracotta Is Oh Yeah: Did I Mention It is JVM-Level? Before Terracotta After Terracotta ID Monitor ID Monitor Object Object Primitive Method() Primitive Method() Field (value) Field (value) Object Object Field (ref) Field (ref) ID Monitor ID Monitor Confidential – for information of designated recipient only. Copyright Terracotta 2006
  22. 22. Performance + Reliability + Simplicity   10X throughput over conventional APIs –  All Reads from memory (implicit locality) –  All Writes are deltas-only –  Statistics and heuristics (greedy locks)   Terracotta Active Server Array –  Simple form available today for enterprise customers –  V1.0 GA this year   Looks like Java to me (code like your mom used to make) –  Normal access patterns: no check-out before view and check-in on commit –  Code and test at a unit level without infrastructure intruding on app logic –  Threads on multiple JVMs look like threads on the same JVM Confidential – for information of designated recipient only. Copyright Terracotta 2006
  23. 23. What Terracotta Is Batching and Ordering In Accordance With JMM   Durable   Transactional   Clustered   Shared Memory   Use synchronized or util.concurrent or Terracotta Integration Module to lock Confidential – for information of designated recipient only. Copyright Terracotta 2006
  24. 24. What Terracotta Is What TC Does for You   Pure Java –  Shares your objects and framework internals –  Shares thread coordination   Efficient Scale-out through heuristics and statistics (what is resident where)   Availability in the box Confidential – for information of designated recipient only. Copyright Terracotta 2006
  25. 25. What Terracotta Is Reduces Clustering to Tuning   Always coherent   Always on disk at in-memory speed (streaming, not seeking)   Always Pure Java (java.*, synchronized, and util.concurrent)   Changes needed usually for tuning Confidential – for information of designated recipient only. Copyright Terracotta 2006
  26. 26. When to use Terracotta: Will It Work With My App? (The Terracotta API, so to speak) Confidential – for information of designated recipient only. Copyright Terracotta 2006
  27. 27. When to Use Terracotta Users and Use Cases   Web Apps   Customer Service –  Portals –  CRM apps –  Social networking –  Rewards / affinity programs –  Gaming, betting   Financials   Telco / IT –  Trading apps –  VOIP –  Trading bus –  Server mgmt Confidential – for information of designated recipient only. Copyright Terracotta 2006
  28. 28. When to Use Terracotta Terracotta is a Platform (not an app server)   How do you share objects –  HashMap, CHM, EHCache, HashTable, TreeMap   How do you coordinate access –  Wait / Notify, synchronized, util.concurrent   Where do you store the data –  In Terracotta as objects, write-behind to DB, write-thru to DB, etc.   How do you process async workloads –  Queue of changes, eventing pattern, etc. Confidential – for information of designated recipient only. Copyright Terracotta 2006
  29. 29. When to Use Terracotta API: Using Patterns That Are Clusterable   Conversation clustering / persistence   Window onto large dataset   Data as a service   Cluster membership and management   Change Notification   Shared Metadata Confidential – for information of designated recipient only. Copyright Terracotta 2006
  30. 30. When to Use Terracotta Pattern: Conversation Clustering   Set a conversation key (cookie)   Value should be object graph –  not primitives like string or byte array   Load Balancer required –  Sticky load balancing (layer 7 preferred, or Apache mod_jk) –  Ensures locality of reference   No DB required Ref. impl. code available now: http://svn.terracotta.org/svn/forge/projects/exam/ Confidential – for information of designated recipient only. Copyright Terracotta 2006
  31. 31. Account Creation   SignupController –  UserValidator –  RegistrationService (DefaultRegistrationService) @RequestMapping(method = RequestMethod.POST) public ModelAndView processSubmit(final HttpServletRequest request, @ModelAttribute final User user, final BindingResult result, final SessionStatus status) throws RegistrationException, ProtocolException { validator.validate(user, result); if (result.hasErrors()) { return new ModelAndView(quot;registration/signupquot;); } else { // make sure that no clear text passwords are stored in the back-end user.setAndEncodePassword(user.getPassword()); // hold the user's registration until it has been confirmed over email this.service.holdForEmailConfirmation(user, UrlHelper.createAbsoluteUrl(request, ConfirmController.class)); status.setComplete(); return new ModelAndView(quot;registration/successquot;, quot;userquot;, user); } } Confidential – for information of designated recipient only. Copyright Terracotta 2006
  32. 32. Account Creation: RegistrationService DefaultRegistrationService public class DefaultRegistrationService implements RegistrationService { private final Configuration freemarkerConfiguration; private final UserService userService; private final MailSender mailSender; @Root private final Map<String, Long> heldUsers = new ConcurrentHashMap<String, Long>(); //... public void holdForEmailConfirmation(final User user, final String confirmationUrl) throws RegistrationException { // ... Validate args final String uuidString = SecurityHelper.generateUniqueCode(); heldUsers.put(uuidString, user.getId()); // ... Code to send confirmation email } } Confidential – for information of designated recipient only. Copyright Terracotta 2006
  33. 33. Account Creation   Durable Java data structure as temporary storage for confirmation codes   Benefits –  No round trip to the database –  No need for caching –  No cache freshness issues (no opportunity for arbitrage) –  No impedance mismatch at O/R layer   Issues –  Must clean up data structure in a cluster-aware way (similar to HTTP Session cleanup)   Password reset function works the same way Confidential – for information of designated recipient only. Copyright Terracotta 2006
  34. 34. AuthC/AuthZ   Spring Security TIM –  Authentication (AbstractAuthenticationToken) and authorization (GrantedAuthorityImpl) tokens held in HTTP Session via Spring Security <filter> <filter-name>springSecurityFilterChain</filter-name> <filter-class>org.springframework.web.filter.DelegatingFilterProxy</filter-class> </filter> <filter-mapping> <filter-name>springSecurityFilterChain</filter-name> <url-pattern>/*</url-pattern> </filter-mapping> –  Clustered via Terracotta session clustering module –  Spring Security TIM handles all instrumentation configuration for objects in HTTP session   LoginController Confidential – for information of designated recipient only. Copyright Terracotta 2006
  35. 35. AuthC/AuthZ: Spring Security Confidential – for information of designated recipient only. Copyright Terracotta 2006
  36. 36. AuthC/AuthZ: LoginController @Controller @RequestMapping(quot;/login.doquot;) public class LoginController { @RequestMapping(method = RequestMethod.GET) public ModelAndView loginHandler(final HttpServletRequest request, final HttpServletResponse response, final HttpSession session) { final ModelAndView result = new ModelAndView(quot;user/loginquot;); final boolean loginError = request.getParameter(quot;login_errorquot;) != null; if (loginError) { result.addObject(quot;loginErrorquot;, loginError); final String lastUser = (String) session.getAttribute(AuthenticationProcessingFilter.SPRING_SECURITY_LAST_USERNAME_KEY); final Throwable lastException = (Throwable)session.getAttribute(AbstractProcessingFilter.SPRING_SECURITY_LAST_EXCEPTION_KEY); if (lastUser != null) { result.addObject(quot;lastUserquot;, lastUser); } if (lastException != null) { result.addObject(quot;errorMessagequot;, lastException.getMessage()); } } return result; } } Confidential – for information of designated recipient only. Copyright Terracotta 2006
  37. 37. Conversation State: Take The Exam   Spring WebFlow –  Workflow through examination process   ExamService (CachingWrapperExamService) –  Caches exam meta-data (questions, answers, sections, etc.)   ExamSessionService –  Manages the state of in-progress examinations using clustered: –  ReentrantReadWriteLock –  Map Confidential – for information of designated recipient only. Copyright Terracotta 2006
  38. 38. Conversational State: List Available Exams <view-state id=quot;chooseExamquot; view=quot;exam/listquot;> <on-render> <evaluate expression=quot;examService.getAllExams()quot; result=quot;viewScope.examsquot; /> </on-render> <transition on=quot;selectExamquot; to=quot;examSelectedquot;> <set name=quot;flowScope.examIdquot; value=quot;requestParameters.examIdquot;></set> </transition> <transition on=quot;returnquot; to=quot;returnquot; /> </view-state> Confidential – for information of designated recipient only. Copyright Terracotta 2006
  39. 39. this.cacheManager = CacheManager.getInstance(); this.cache = this.cacheManager.getEhcache(EXAM_CACHE_NAME); } Conversational State: View Exam Details <view-state id=quot;examSelectedquot; view=quot;exam/detailsquot;> <on-render> <evaluate expression=quot;examService.findById(flowScope.examId)quot; result=quot;viewScope.examquot;/> </on-render> <transition on=quot;backquot; to=quot;chooseExamquot; /> <transition on=quot;returnquot; to=quot;returnquot; /> <transition on=quot;startExamquot; to=quot;startExamquot; /> </view-state> public class CachingWrapperExamService implements ExamService { /* as defined in ehcache.xml */ private static final String EXAM_CACHE_NAME = quot;examCachequot;; private final CacheManager cacheManager; private final Ehcache cache; private final ExamService examService; public CachingWrapperExamService(ExamService examService){ this.examService = examService; this.cacheManager = CacheManager.getInstance(); this.cache = this.cacheManager.getEhcache(EXAM_CACHE_NAME); } // ... public Exam findById(Long id) { Exam exam = getCached(id); if (exam == null){ exam = cache(this.examService.findById(id)); } return exam; } // ... } Confidential – for information of designated recipient only. Copyright Terracotta 2006
  40. 40. Conversation State: Start Exam <action-state id=quot;startExamquot;> <on-entry> <evaluate expression=quot;examSessionService.startExam(currentUser.name, requestParameters.examId)quot;/> </on-entry> <evaluate expression=quot;examSessionService.getFirstQuestion(currentUser.name).getId()quot; result=quot;flowScope.questionIdquot;/> <transition to=quot;showQuestionquot;/> </action-state> public class ExamSessionServiceImpl implements ExamSessionService { private static final Logger logger = Logger.getLogger(ExamSessionServiceImpl.class); // a map storing userName -> examSession mapping of currently active exams @Root private final Map<String, ExamSession> ongoingExams = new HashMap<String, ExamSession>(); private final ScheduledExecutorService examTimeoutExecutor = Executors.newScheduledThreadPool(1); private final Map<String, Future> scheduledTimeOutTasks = new HashMap<String, Future>(); @Root private final ReentrantReadWriteLock rwLock = new ReentrantReadWriteLock(); private final ExamService examService; private final UserService userService; @Autowired public ExamSessionServiceImpl(final ExamService examService, final UserService userService) { this.examService = examService; this.userService = userService; } // ... } Confidential – for information of designated recipient only. Copyright Terracotta 2006
  41. 41. Conversation State: Start Exam (cont.) public class ExamSessionServiceImpl implements ExamSessionService { // ... public ExamSession startExam(final String userName, final Long examId) throws ExamException { final User user = userService.findByUserName(userName); ExamSession prev = null; try { rwLock.writeLock().lock(); prev = ongoingExams.get(userName); if (prev != null && prev.getExam().getId().longValue() != examId.longValue()) { throw new ExamAlreadyInProgressException(userName, prev.getExam(), quot;Exam already in progress for user: quot; + userName); } // return previously ongoing exam if attempted to start same exam again if (prev != null) return prev; final Exam exam = examService.findById(examId); final ExamSession session = new ExamSession(); session.setUser(user); session.setExam(exam); ongoingExams.put(userName, session); // run the timeout after 2 secs from the actual time so that exam time out happens on the client first // and give the user a chance to get ExamTimedOutException when exam times out on client final Future timeoutTask = examTimeoutExecutor.schedule(new ExamTimeoutTask(this, userName), session .getRemainingTimeInSeconds() + 2, TimeUnit.SECONDS); scheduledTimeOutTasks.put(userName, timeoutTask); return session; } finally { rwLock.writeLock().unlock(); } } // ... } Confidential – for information of designated recipient only. Copyright Terracotta 2006
  42. 42. Coherent View Of Global Data: View Ongoing Exams @Controller @RequestMapping(quot;/exam/ongoing.doquot;) @RolesAllowed( { StandardAuthoritiesService.ADMINISTRATOR }) public class OngoingExamsController { private final ExamSessionService service; @Autowired public OngoingExamsController(final ExamSessionService examSessionService) { this.service = examSessionService; } @RequestMapping(method = RequestMethod.GET) public ModelAndView listOngoingExams() { final ModelAndView result = new ModelAndView(quot;exam/ongoingquot;); result.addObject(quot;examSessionsquot;, service.getOngoingExams()); return result; } protected OngoingExamsController() { // protected default constructor is needed for CGLib AOPx service = null; } } Confidential – for information of designated recipient only. Copyright Terracotta 2006
  43. 43. Coherent View Of Global Data: View Ongoing Exams public class ExamSessionServiceImpl implements ExamSessionService { private static final Logger logger = Logger.getLogger(ExamSessionServiceImpl.class); // a map storing userName -> examSession mapping of currently active exams @Root private final Map<String, ExamSession> ongoingExams = new HashMap<String, ExamSession>(); private final ScheduledExecutorService examTimeoutExecutor = Executors.newScheduledThreadPool(1); private final Map<String, Future> scheduledTimeOutTasks = new HashMap<String, Future>(); @Root private final ReentrantReadWriteLock rwLock = new ReentrantReadWriteLock(); private final ExamService examService; private final UserService userService; // ... public Collection<ExamSession> getOngoingExams() { Collection<ExamSession> values = null; try { rwLock.readLock().lock(); values = new ArrayList(ongoingExams.values()); } finally { rwLock.readLock().unlock(); } return values; } // ... } Confidential – for information of designated recipient only. Copyright Terracotta 2006
  44. 44. Conversation State And Coherent View Of Global Data   Conversation State –  Memory-speed access to flow data –  Coherent cluster-wide –  Database-like durability for failover –  Handles failure of any/every node –  Completely transparent: no put-back on change   Coherent View Of Global Data –  Coherent cluster-wide –  No round-trip to database »  Ergo, no database abuse »  Ergo, no caching »  Ergo, no cache freshness probs –  Memory-speed read locks –  Simple data model »  ReentrantReadWriteLock for global thread access control »  Map holds global data –  Need to refine large data set access pattern »  @ 20K ongoing exams, single-page access is non-sensical Confidential – for information of designated recipient only. Copyright Terracotta 2006
  45. 45. Case Study: Conversation Clustering Confidential – for information of designated recipient only. Copyright Terracotta 2006
  46. 46. Large Publisher Gets Caught Down the Path with Oracle Scaling Out or Up? Confidential – for information of designated recipient only. Copyright Terracotta 2006
  47. 47. Breaking the Pattern without Leaving “Load-Balanced” World • $1.5 Million DB & HW savings • Doubled business • More than halved database load Confidential – for information of designated recipient only. Copyright Terracotta 2006
  48. 48. Results   Database was still the SoR which kept reporting and backup simple   Scalability was increased by over 10X   Availability was not compromised since test data was still on disk, but in memory-resident format instead of relational   …simple scalability + availability Confidential – for information of designated recipient only. Copyright Terracotta 2006
  49. 49. When to Use Terracotta Pattern: Window Onto Large Dataset   Isolate the JVMs from each other carefully –  ConcurrentHashMap of ConcurrentHashMaps –  One map per L1 JVM   If you must lock while updating, lock the value graph itself (fine- grained) using ReadWriteLock   Enough Heap in each JVM to hold most data it needs –  High pre-fetch rate   You must come up with a way to load balance or partition or this will thrash the network Confidential – for information of designated recipient only. Copyright Terracotta 2006
  50. 50. When to Use Terracotta Pattern: Data As A Service (impl coming December)   Service Oriented   Write a stateless get() router with ActiveMQ –  Send all messages to the get() router –  He hashes and sends the get() to appropriate stripe –  Transparent to service consumer   Fairly flat key / value pairs –  String or Integer key –  Value == shallow object graph (mostly primitives)   Key must hash evenly. Don’t hash spatially even though it is tempting. (e.g., “New York customers” is worse than “A”)   Use MySQL to index keyspace and execute queries that return vectors of key/value pairs to load   Update MySQL write-behind (another pattern) Confidential – for information of designated recipient only. Copyright Terracotta 2006
  51. 51. When to Use Terracotta Pattern: Cluster Membership   Want to know when L1 JVMs come and go –  Listen for Terracotta JMX events –  Catch L1 disconnect event and deterministically take over work (next worker ID after failed worker)   Want to know when L2 JVMs fail over –  Listen for Terracotta JMX events –  If you don’t reconnect after certain time, system.exit() and use initd to restart yourself –  Avoid disconnected mode   Always use a surviving L1 JVM to address a departing one. Do not use System exit hooks! Clean up on restart if last JVM in the cluster   Don’t requeue work, steal queue references instead Confidential – for information of designated recipient only. Copyright Terracotta 2006
  52. 52. When to Use Terracotta Pattern: Change Notification (impl coming soon)   Use this one for write-behind   Linked Blocking Queue in each L1 JVM   Share the LBQ via DSO in a CHM of queues   Have your changeable objects implement an iChangeable interface. Call flush() method on change. You will have to implement this method   Daemon thread takes batches of object references and calls flush on each.   Idempotency is req’d to avoid transactions / JTA. Then peek instead of take and take after flush().   Use Cluster membership (another pattern) to recover from L1 JVM failure Confidential – for information of designated recipient only. Copyright Terracotta 2006
  53. 53. When to Use Terracotta Pattern: Shared Metadata   Composite Pattern…   Use EHCache + Linked Blocking Queue   Conversation Clustering pattern   Change Notification pattern Confidential – for information of designated recipient only. Copyright Terracotta 2006
  54. 54. Summary   Simplicity, scalability, and availability can be friends   Write normal Java code that works across JVMs   Use clustered architecture patterns –  Conversation clustering/persistence –  Window onto large dataset –  Change notification –  Shared metadata –  Clustered membership and data management   Leave business data in the database, use durable NAM for application state data   Centralized operational control: manage your application cluster like you do your database   Terracotta is open source –  Free to use through production –  Enterprise subscription, training, and services available Confidential – for information of designated recipient only. Copyright Terracotta 2006
  55. 55. Resources   Open Source (MPL-based) JVM-level clustering: http://www.terracotta.org   Apress / Amazon.com: “Definitive Guide to Terracotta” –  By Alex Miller, Ari Zilka, Geert Bevin, Jonas Bonér, Orion Letizi, Taylor Gautier   Forums: http://forums.terracotta.org/   Enterprise Offerings: http://www.terracottatech.com/ Confidential – for information of designated recipient only. Copyright Terracotta 2006

×