Session Code: DAT451    Designing Role-Based Database    Systems to Achieve Unlimited    Database Scalability    Fernando ...
What forced me to prepare this session5
The "unpredictable workload" case     Millions of copies of a win app around the     world     All of them waiting for fil...
The "unreasonable scalability" case     The system already reached its limits     They need to support workload 50 times h...
Introductions     Fernando G. Guerrero     CEO of Solid Quality Global LLC, USA     President of the SolidQ subsidiaries a...
The SolidQ recipe      4 SQL Server MVPs old      friends speaking at a      conference in Palm Springs      A pleasant di...
What are you going to see and hear today10
This session is about…      Lesson learned from “unreasonable” and      amazing real systems from real enterprise      cus...
This session is NOT about…      Code      Demos      Obscure trace flags and registry hacks      Any particular commercial...
13
First… Listen14
Then, try to understand what        their needs really are…      And what they really want15
A database system is just one      more tool to help achieving            business goals16
Your technical common sense is        your most powerful tool17
If you don’t need to buy       anything to implemnent a     solution, it does not mean that      this is the cheapest solu...
Role based DB system design      A single box might be sufficient      Different tasks might imply different roles      Ro...
The unpredictable workload case20
Role based DB system design      A single box might be sufficient      Different tasks might imply different roles      Ro...
Massive attack22
The unpredictable workload case23
A common sense powered queue      Multi-layered server roles:        The Reader        The Writer        The Master Root  ...
The Magic Array25
The Reader role      Organized as a Readers tree with a Reader Root      at the top      Verifies signature of requested f...
The Writer role      Works as a Reader as well      Organized as a flexible array of Writers      Sends signature to the M...
The Master Root      Stores new files and signatures      Calls the qualification engine to qualify new      files      Di...
Massive dynamic deployment29
The unreasonable scalability case30
Scalability needs Jan 2005                    Workload vs Capacity                Workload   Initial C   Improved C       ...
32      Jan-05                10                90                            300     Feb-05     Mar-05     Apr-05     May...
Side note:     is there such thing as a write-            only transaction?                   or…     how much writing act...
Scalability needs Dec 2008                                              Workload vs Capacity                          Work...
Scalability needs beyond 2009                                                     Workload vs Capacity                    ...
The unreasonable scalability case36
Role-based architecture37
Detailed role assignment38
Technology alone cannot beat     what an experienced engineer      can design on the back of an               envelope39
40
Resources for IT Professionalswww.microsoft.com/techedTech·Talks        Tech·Ed BloggersLive Simulcasts   Virtual Labshttp...
Related ContentBreakout Sessions (session codes and titles)Interactive Theater Sessions (session codes and titles)Hands-on...
Track ResourcesResource 1Resource 2Resource 3Resource 4
1 Year           Subscription!Complete anevaluation onCommNet andenter to win!
© 2008 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be...
Designing Role-Based Database Systems to Achieve Unlimited Database Scalability
Upcoming SlideShare
Loading in …5
×

Designing Role-Based Database Systems to Achieve Unlimited Database Scalability

317 views

Published on

Published in: Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
317
On SlideShare
0
From Embeds
0
Number of Embeds
2
Actions
Shares
0
Downloads
5
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide
  • Animation still to be finished
  • Animation still to be finished
  • Animation still to be finished
  • Designing Role-Based Database Systems to Achieve Unlimited Database Scalability

    1. 1. Session Code: DAT451 Designing Role-Based Database Systems to Achieve Unlimited Database Scalability Fernando G. Guerrero CEO Solid Quality Mentors2
    2. 2. What forced me to prepare this session5
    3. 3. The "unpredictable workload" case Millions of copies of a win app around the world All of them waiting for file safety, before using them At undefined times Unlimited number of potential files to check Unpredictable level of workload at any date and time Unpredictable geographical distribution of workload Potentially, millions of requests per hour Can we solve it?6
    4. 4. The "unreasonable scalability" case The system already reached its limits They need to support workload 50 times higher Scaling up is not an option Scaling out is not a simple task They need to support over 15K transactions per second, with textual based data Can we solve it?7
    5. 5. Introductions Fernando G. Guerrero CEO of Solid Quality Global LLC, USA President of the SolidQ subsidiaries around the World fguerrero@solidq.com Microsoft Regional Director for Spain since 2004 SQL Server MVP from year 2000 till 2007 Usual suspect at many international conferences8
    6. 6. The SolidQ recipe 4 SQL Server MVPs old friends speaking at a conference in Palm Springs A pleasant dinner at a nice restaurant 2 bottles of my favorite Spanish wine 1 bottle of a great single Malt Whisky Lots of honest and trustful9 work for 6 years
    7. 7. What are you going to see and hear today10
    8. 8. This session is about… Lesson learned from “unreasonable” and amazing real systems from real enterprise customers Based on actual work performed by our Solid Quality mentors Learning to think out of the box when designing DB systems Thinking first, acting later11
    9. 9. This session is NOT about… Code Demos Obscure trace flags and registry hacks Any particular commercial product or service offering Description about standard scalability techniques12
    10. 10. 13
    11. 11. First… Listen14
    12. 12. Then, try to understand what their needs really are… And what they really want15
    13. 13. A database system is just one more tool to help achieving business goals16
    14. 14. Your technical common sense is your most powerful tool17
    15. 15. If you don’t need to buy anything to implemnent a solution, it does not mean that this is the cheapest solution18
    16. 16. Role based DB system design A single box might be sufficient Different tasks might imply different roles Roles might be incompatible You’ll never know until you measure it19
    17. 17. The unpredictable workload case20
    18. 18. Role based DB system design A single box might be sufficient Different tasks might imply different roles Roles might be incompatible You’ll never know until you measure it21
    19. 19. Massive attack22
    20. 20. The unpredictable workload case23
    21. 21. A common sense powered queue Multi-layered server roles: The Reader The Writer The Master Root “Magic array” of virtual server names Self updated definitions Proprietary advanced hash-based checking Local vs remote storage24
    22. 22. The Magic Array25
    23. 23. The Reader role Organized as a Readers tree with a Reader Root at the top Verifies signature of requested file Replies to client with simple response: Safe Unsafe Unknown If unknown, checks parent Reader for further information Receives updates from parent Reader26
    24. 24. The Writer role Works as a Reader as well Organized as a flexible array of Writers Sends signature to the Master Root and sends file asynchronously if necessary Replies immediately to client with simple response: Safe Unsafe Unknown Receives updates from Distributor27
    25. 25. The Master Root Stores new files and signatures Calls the qualification engine to qualify new files Distributes updates to root Reader Distributes updates to array of Writers28
    26. 26. Massive dynamic deployment29
    27. 27. The unreasonable scalability case30
    28. 28. Scalability needs Jan 2005 Workload vs Capacity Workload Initial C Improved C 300 300 90 100 90 1031
    29. 29. 32 Jan-05 10 90 300 Feb-05 Mar-05 Apr-05 May-05 Jun-05 Jul-05 Workload Aug-05 Sep-05 Oct-05 Nov-05 Initial C Dec-05 90 200 300 Jan-06 Feb-06 Workload vs Capacity Mar-06 Improved C Apr-06 Scalability needs Aug 2006 May-06 Jun-06 Jul-06 90 300 600 Aug-06
    30. 30. Side note: is there such thing as a write- only transaction? or… how much writing activity do you really expect?33
    31. 31. Scalability needs Dec 2008 Workload vs Capacity Workload Initial C Improved C Role Based C 4000 3000 2000 1500 600 600 600 600 200 300 500 90 10 90 90 90 90 90 Apr-05 Apr-06 Apr-07 Apr-08 Jan-06 Jan-05 Jan-07 Jan-08 Jul-05 Oct-05 Jul-06 Oct-06 Jul-07 Oct-07 Jul-08 Oct-0834
    32. 32. Scalability needs beyond 2009 Workload vs Capacity Workload Initial C Improved C Role Based C 16000 12000 8000 6000 4000 3000 2000 1500 90 10 200 90 600600 300 500 90 90 600 90 600 90 600 90 600 90 Sep-05 Sep-06 Sep-07 Sep-08 Sep-09 Sep-10 Jan-05 Jan-06 Jan-07 Jan-08 Jan-09 Jan-10 May-10 May-05 May-06 May-07 May-08 May-0935
    33. 33. The unreasonable scalability case36
    34. 34. Role-based architecture37
    35. 35. Detailed role assignment38
    36. 36. Technology alone cannot beat what an experienced engineer can design on the back of an envelope39
    37. 37. 40
    38. 38. Resources for IT Professionalswww.microsoft.com/techedTech·Talks Tech·Ed BloggersLive Simulcasts Virtual Labshttp://microsoft.com/technetEvaluation licenses, pre-releasedproducts, and MORE!
    39. 39. Related ContentBreakout Sessions (session codes and titles)Interactive Theater Sessions (session codes and titles)Hands-on Labs (session codes and titles)Hands-on Labs (session codes and titles)
    40. 40. Track ResourcesResource 1Resource 2Resource 3Resource 4
    41. 41. 1 Year Subscription!Complete anevaluation onCommNet andenter to win!
    42. 42. © 2008 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.46

    ×