The document describes the SEDA (Staged Event-Driven Architecture) framework. SEDA divides applications into stages connected by queues. Each stage has a thread pool and dynamic resource controllers that adjust thread allocation to meet performance targets. This allows applications built with SEDA to scale responsive and robustly handle varying load. SEDA was implemented in a system called Sandstorm that provides APIs for building networked services using asynchronous I/O and event-driven programming. Example services implemented with SEDA demonstrated improved throughput and response times under heavy load compared to traditional concurrency approaches.

1. SEDA: An Architecture for Well-
Conditioned, Scalable Internet Services
by Welsh, Culler, and Brewer
UC San Diego
CSE 294
Winter Quarter 2007
Barry Demchak

2. 2
The Objective
 Services that support millions of users
 Responsive
 Robust
 Highly available
 Support large swings in load over time (100x)
 General purpose mechanisms

3. 3
A Web Application Example: DNS
 Verisign’s root servers
 Two 10-high stacked 1U IBM eServers
running Solaris and Red Hat Linux
 26 billion requests/day (normal traffic) or
300K-500K requests per second
 In 2000, there were 1 billion requests/day
 In 2010, predicting 200 billion requests/day
-- InfoWorld, February 16, 2007, DNS Attack
Puts Web Security in Perspecitive,
Roger Grimes

4. 4
SEDA
 Problem Background
 Survey of existing alternatives
 SEDA design
 Implementation results
 Applicability to ESBs (e.g., Mule)
 Applicability to THE picture

5. 5
The Environment
 Static content becoming dynamic content
 Rapidly changing service logic
 Hosting on general purpose platforms

6. 6
Non-Solutions
 Replication
 Traditional operating systems
 Traditional concurrency

7. 7
Solution
 SEDA = Staged Event-Driven Architecture
 Applications decompose to stages
 Stages have incoming event queues
 Dynamic resource throttling

8. 8
Definitions
 Well-conditioned
 Behaves like a simple pipeline: output rate
scales to input rate
 Excessive demand does not degrade pipeline
throughput
 Graceful degradation
 Under load, response time degrades linearly
with length of queue
 Degraded response time constant for all
clients (subject to service policies)

9. 9
The Existing Choices
 Thread-based Concurrency
 Bounded Thread Pools
 Event-driven Concurrency
 Structured Event Queues

10. 10
Thread-based Concurrency
 Thread-per-request model (RPC, JMI,
DCOM)

11. 11
Thread-based Performance

12. 12
Bounded Thread Pools
 Initial thread pool subject to expansion
 Fixed thread pools cause unfairness to clients
because of long waits when threads are
blocked
 Policies of reordering/prioritizing threads
based on expense of request are difficult

13. 13
Event-driven Concurrency
 Queue-based scheduling of threads
 Threads execute finite state machines

14. 14
Event-driven Performance

15. 15
Structured Events Queues
 Sets of event queues

16. 16
SEDA Goals
 Support massive concurrency
 Simplify construction of well-conditioned
services
 Enable introspection
 Support self-tuning resource management

17. 17
SEDA Terms
 Stage = fundamental processing unit
 Event handler
 Incoming event queue
 Thread pool
 Controller = scheduler for a stage

18. 18
A SEDA Application – Web server

19. 19
Dynamic Resource Controllers
 Shields programmers from performance
tuning
 Observes stage’s runtime characteristics, and
adjusts allocation and scheduling parameters
to meet performance targets

20. 20
Dynamic Resource Controllers

21. 21
Thread Pool Controller Performance

22. 22
Batching Controller Performance

23. 23
Implementation: Sandstorm
 Internet-based service platform
 Each application implements handleEvents()
 APIs for:
 Naming, creating, destroying stages
 Performing queue operations
 Controlling queue thresholds
 Profiling and debugging
 Asynchronous I/O package
 7900 lines of Java

24. 24
Asynchronous I/O

25. 25
Asynchronous I/O Performance

26. 26
Application and Evaluation
 Haboob: a high performance HTTP server
 Benchmarks
 Architecture
 Adaptive load shedding
 Gnutella packet router
 Architecture

27. 27
Haboob Throughput

28. 28
Haboob Response Time

29. 29
SEDA Summary
 Establishes principles toward Internet-style
operating environments
 Stages ease concurrency complexity and
encourage modularity
 Dynamic controllers enable novel scheduling
and resource management strategies
 Challenges: detecting cause of overload
conditions, and control strategy to cure
overload

30. 30
Mule
 From “Implementing an ESB using Mule” by
Ross Mason for JavaZone 2005
http://mule.mulesource.org/wiki/download/attachments/223/javazone-2005-mule-real-world-old.ppt?version=1
 ESB
 Loosely coupled components
 Event driven
 Highly distributed
 Intelligent Routing
 Data transformation
 Multiprotocol message bus …

31. 31© 2005
Mule Topologies
Enterprise Service Bus
Client/Server and Hub n' Spoke
Peer Network
Pipeline
Enterprise Service Network

32. 32
Messaging – JMS Style
 JMS = Java Messaging Service
 Connection-oriented API exposes point-to-
point and publish/subscribe communications
 Queue-oriented

33. 33© 2005
Loan Broker Design

34. 34© 2005
Design With Mule