Your SlideShare is downloading. ×
0
Service Dispatcher: The MySpace
Implementation of Service Broker
       Speaker: Christa Stelzmuller
      MySpace Chief D...
Christa Stelzmuller

  Chief Data Architect at MySpace since Oct 2006
  Formerly at Yahoo!
    Engineering Manager
    Dat...
What is Service Broker?

Message Queue
  Asynchronous
  Unicast
  Intra- and Inter-database, instance, and server
Integrat...
Why Service Broker?

Flexible
  Atomic transactions can be split across synchronous and
  asynchronous work items
  Work i...
Challenges with Service Broker

Unicast
  Communication is from a single sender to a single target
  Excessive message tra...
Introducing Service Dispatcher
Core Features
 Simulated Multicast
   Communication is from a single sender to a specified set of
   targets
   Wildcard t...
Advanced Features
 Dialog Management
   Technique of dialog reuse abstracted
   Pooling based on spid, service, and group
...
Major Use Cases
  Broker Gateway
    Asynchronous message
    delivery APIs
    Load balanced farm
    Abstracted data des...
Major Use Cases

Backend Initiated
Processing
  Foreign keys and
  relationships spread across
  many databases
  Coordina...
Major Use Cases

List Broadcast
  Local data used to broadcast
  updates to lists
  Two views of the relationship
  repres...
Developer Highlights

Message Types
  No need for explicit service broker message types
  If needed, application message t...
Developer Highlights

Conversation Group Locking
  Alternative to GET CONVERSATION GROUP
  Dialog consolidation for perfor...
Administrator Highlights

Build
  Central catalog
  Common client code deployed
  Automated services
    Dispatcher route ...
Administrator Highlights

Health & Monitoring
  Reports
    Scheduled
    Ad-Hoc
  Notifications
  Automated queue reactiv...
Unsupported Broker Features

Message Ordering
  Farm of servers, random access
  Dialog death & resend
  Abstracted conver...
Upcoming Features

Dialog Emulation
  Introducing the concept of circuits
  Message sequence number
  Message ordering bec...
Service Broker Stats
                     Average        Average Peak          Max          Peak Farm Total
    MySpace DB...
When not to use Dispatcher

Local asynchronous messages
One-to-one messaging
When an unsupported feature is required
Alternatives to Dispatcher

Abstracted SEND logic (same as dispatcher without the
router component)
Dynamic route manageme...
More Information

Case Study
  http://www.microsoft.com/casestudies/Case_Study_Detail.
  aspx?CaseStudyID=4000004532


Cod...
Credits

Creator & Original Concept, Mikhail Berlyant
Principal Dispatcher Architect, Mark Lovretovich
Contributors, Kevin...
To learn more or inquire about speaking opportunities, please
                           contact:

Mark Ginnebaugh, User G...
Upcoming SlideShare
Loading in...5
×

MySpace SQL Server Service Broker

4,749

Published on

Service Dispatcher: The MySpace Implementation of Service Broker

Christa Stelzmuller, Chief Data Architect at MySpace, speaking to the San Francisco SQL Server User Group in October 2009. http://bit.ly/cTgAH

Christa explains the architecture and code used to power Service Dispatcher, the broker based component MySpace built to enhance the broker feature set. This presentation covers:

•MySpace use cases leading to Service Broker as the choice for ETL and distributed transaction management.

•How MySpace handles data integrity in a distributed environment.

•Technical details of MySpace's Service Dispatcher component.

•How you can simplify your Service Broker applications.

Published in: Technology
1 Comment
5 Likes
Statistics
Notes
No Downloads
Views
Total Views
4,749
On Slideshare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
96
Comments
1
Likes
5
Embeds 0
No embeds

No notes for slide

Transcript of "MySpace SQL Server Service Broker "

  1. 1. Service Dispatcher: The MySpace Implementation of Service Broker Speaker: Christa Stelzmuller MySpace Chief Data Architect San Francisco SQL Server User Group September 14, 2009 Mark Ginnebaugh, User Group Leader www.bayareasql.org
  2. 2. Christa Stelzmuller Chief Data Architect at MySpace since Oct 2006 Formerly at Yahoo! Engineering Manager Data Architect for the Yahoo! Music Team Specializes in very large databases with high volumes of transactions Tonight’s Topic: Service Dispatcher: The MySpace Implementation of Service Broker
  3. 3. What is Service Broker? Message Queue Asynchronous Unicast Intra- and Inter-database, instance, and server Integrated into the database engine Instance level DML extensions
  4. 4. Why Service Broker? Flexible Atomic transactions can be split across synchronous and asynchronous work items Work items can occur in parallel Predictable Asynchronous work items are processed at a constant rate leveling peaks and valleys Transactional In-Database Common administration
  5. 5. Challenges with Service Broker Unicast Communication is from a single sender to a single target Excessive message transmission Routing Requires a physical route between each service Unmanageable number of routes Non-Intuitive Four object types must be implemented for each service, and up to seven understood High learning curve for average database developer
  6. 6. Introducing Service Dispatcher
  7. 7. Core Features Simulated Multicast Communication is from a single sender to a specified set of targets Wildcard targets support simulated broadcast Route Management Routes to all dispatcher client enabled databases are managed centrally Abstraction API for “SEND” Friendly database names are used instead of service broker instances
  8. 8. Advanced Features Dialog Management Technique of dialog reuse abstracted Pooling based on spid, service, and group Route Receipts Used in simulated broadcast scenarios to return information on all servers from which acknowledgement should be expected Extensible Publish/Subscribe (target agnostic messages) Notifications
  9. 9. Major Use Cases Broker Gateway Asynchronous message delivery APIs Load balanced farm Abstracted data destination Example Usage Activities Stream
  10. 10. Major Use Cases Backend Initiated Processing Foreign keys and relationships spread across many databases Coordinated messaging Example Usage Profile management Content take-downs
  11. 11. Major Use Cases List Broadcast Local data used to broadcast updates to lists Two views of the relationship represented Example Usage Photo Tags Friend Categories
  12. 12. Developer Highlights Message Types No need for explicit service broker message types If needed, application message types are supported as part of the message body Standard set of message types, system and dispatcher Contracts No need for explicit service broker contracts Each application service must contain a reference to at least one dispatcher contract
  13. 13. Developer Highlights Conversation Group Locking Alternative to GET CONVERSATION GROUP Dialog consolidation for performance Requires explicit conversation group lock release Conversation Ending Dispatcher specific state tables Error handling
  14. 14. Administrator Highlights Build Central catalog Common client code deployed Automated services Dispatcher route creation Client route creation Database Configuration GET CONVERSATION GROUP Maximum valid groups Dialog pool size Dialog timeout settings
  15. 15. Administrator Highlights Health & Monitoring Reports Scheduled Ad-Hoc Notifications Automated queue reactivation Quarantine Ping
  16. 16. Unsupported Broker Features Message Ordering Farm of servers, random access Dialog death & resend Abstracted conversation management Fire & Forget Conversation management is tightly controlled by dispatcher, and logic to properly handle conversation ends and errors is built in. Begin/End Conversation Applications cannot control dialog state, this is entirely managed by dispatcher
  17. 17. Upcoming Features Dialog Emulation Introducing the concept of circuits Message sequence number Message ordering becomes possible Applications can control circuit state Abstracted activation logic RECEIVE logic abstracted Internal broker message type handling
  18. 18. Service Broker Stats Average Average Peak Max Peak Farm Total MySpace DB Sends/Sec/Server Sends/Sec/Server Sends/Sec/Server Sends/Sec Dispatcher 45 229 816 9,164 Profile 90 258 925 130,290 Mail 40 2,397 8,546 1,210,485 Shared 264 540 - - Features 21 36 37 720 Average Average Peak Max Peak Farm Total MySpace DB Rcvs/Sec/Server Rcvs/Sec/Server Rcvs/Sec/Server Receives/Sec Dispatcher 10 33 48 1,328 Profile 60 303 432 153,015 Mail 38 3,186 4,980 1,608,930 Shared 218 351 - - Features 18 92 119 1,840
  19. 19. When not to use Dispatcher Local asynchronous messages One-to-one messaging When an unsupported feature is required
  20. 20. Alternatives to Dispatcher Abstracted SEND logic (same as dispatcher without the router component) Dynamic route management (Microsoft’s solution
  21. 21. More Information Case Study http://www.microsoft.com/casestudies/Case_Study_Detail. aspx?CaseStudyID=4000004532 Codeplex Soon to come!
  22. 22. Credits Creator & Original Concept, Mikhail Berlyant Principal Dispatcher Architect, Mark Lovretovich Contributors, Kevin Stephenson & Justin Engelman
  23. 23. To learn more or inquire about speaking opportunities, please contact: Mark Ginnebaugh, User Group Leader mark@designmind.com
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×