Tutorial on Constructing a Web-Server with Patterns at ADC 2004
Upcoming SlideShare
Loading in...5
×
 

Like this? Share it with your network

Share

Tutorial on Constructing a Web-Server with Patterns at ADC 2004

on

  • 2,145 views

 

Statistics

Views

Total Views
2,145
Views on SlideShare
2,143
Embed Views
2

Actions

Likes
2
Downloads
50
Comments
0

2 Embeds 2

http://www.slideshare.net 1
http://www.techgig.com 1

Accessibility

Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

Tutorial on Constructing a Web-Server with Patterns at ADC 2004 Presentation Transcript

  • 1. Advanced Developers Conference 10.-11. Oktober 2004 Bemerkungen Pra 03 Constructing a Web- Servers with Patterns Michael Stal Siemens Corporate Technology Bemerkungen Michael Stal – Aufbau eines Web- 1 Servers mit Hilfe von Patterns Case Study Bemerkungen A High-Performance Web Server Dein Name – Dein Vortragstitel Pra 01 - S. 1
  • 2. Advanced Developers Conference 10.-11. Oktober 2004 Bemerkungen Web Server: The Domain Web Servers provide content and content delivery services to internet users. Bemerkungen Web Server: Key Challenges Different end user needs and traffic workloads require configurable web server architectures regarding the following aspects: Concurrency models: thread per request,Bemerkungen thread pool, ... Event demultiplexing models: synchronous or asynchronous File caching models: least-recently used (LRU), least-frequently used (LFU), ... Content delivery protocols: HTTP/1.0, HTTP/1.1, HTTP-NG Operating system: UNIX, Win32, ... Dein Name – Dein Vortragstitel Pra 01 - S. 2
  • 3. Advanced Developers Conference 10.-11. Oktober 2004 Bemerkungen Event Demultiplexing (I) A Web server must demultiplex and process multiple types of event, such as CONNECT and HTTP GET requests. To be adaptable towards different usage scenarios, its concrete event demultiplexing and dispatching architecture should: Support changing the event demultiplexing and dispatching code independently of event processing code. Allow the integration of new or improved versions of the event processing services. Support a high throughput. Be simple to use. Bemerkungen Event Demultiplexing (II) Separate event demultiplexing and dispatching from event processing via Reactor: Event handlers (HTTP_Handler) represent the web server‘s services for dispatches processing events that can occur on an * Event_ Bemerkungen Reactor Handler associated event source. maintains owns Handles (Handle) represent event * * sources whose events must be select() Handle notifies processed. An event demultiplexer (select()) HTTP_ HTTP_ Handler 1 Handler 2 discovers and demultiplexes events that occur on handles. A reactor (Reactor) provides the web server‘s event loop and dispatches events that occurred on Handles to their associated HTTP_Handler. Dein Name – Dein Vortragstitel Pra 01 - S. 3
  • 4. Advanced Developers Conference 10.-11. Oktober 2004 Bemerkungen Reactor: Problem Reactor (POSA2) Clients Problem: Event-driven applications must be prepared to receive multiple service requests simultaneously, but to simplify programming, Logging these requests should be processed serially. How Records This is a logging This is a record can we provide such an infrastructure so that: logging record This is a logging record Event an application does not block on any single event Sources source and exclude other event sources from being serviced. Logging Server application developers are shielded from low-level event detection, de-multiplexing, and dispatching mechanisms. an application is extensible with new event handling components. Bemerkungen a high throughput can be achieved. Output Media Reactor: Structure Solution Structure: Separate event dispatches * demultiplexing and dispatching from event Reactor Event Handler processing. handle_event() register() maintains get_handle() unregister() handle_events() The OS provides mechanisms to represent owns event sources (Handles) and to detect and * Bemerkungen Concrete demultiplex events that occur on these event Handle Event Handler * sources (Event Demultiplexers like Event Demuxer select()). void handle_event(Event e) { notifies switch (e) { A Reactor encapsulates the OS event select() case READ_EVENT: // … case WRITE_EVENT: // … demultiplexing mechanisms and provides // … } } functionality to dispatch events to application void handle_events() { // Listen for events to occur. services. result = select(…); // Dispatch event handler. Event Handlers provide functionality to process for (HANDLE h = 0; h < max_Handle; ++h) { if (FD_ISSET (&read_fds, h) certain types of event that occur on a certain handler_table[h].event_handler-> handle_event(READ_EVENT); event source. They register with the reactor to // Same for WRITE and EXCEPT events. } } get dispatched when the respective events occur on their event source. Dein Name – Dein Vortragstitel Pra 01 - S. 4
  • 5. Advanced Developers Conference 10.-11. Oktober 2004 Bemerkungen Reactor: Dynamics Solution Dynamics: Dispatch an event handler whenever an event occurs on its handle for Reactor Handler 1 Handler 2 which it has registered. register(handle, events) The Concrete Event Handlers register their register(handle, events) Handle and all events of interest with the Reactor. handle_events The application’s main event loop calls the select Reactor to listen for events to occur on all registered Handles. Once one or more events occurred on the handle_event(event) Handles, the Reactor serially dispatches their Event Handlers to process the occurred events, if the Handlers had registered for them.Bemerkungen Reactor: Consequences Benefits Separation of concerns: event de-multiplexing and dispatching infrastructure is separated from application- specific event processing. Reusability: the event de-multiplexing and dispatching Bemerkungen infrastructure is a reusable component. Extensibility: new or improved event handlers can be added easily. Ease of use: a Reactor provides a simple, serial event handling architecture. Liabilities Restricted applicability: requires appropriate OS support. Non-pre-emptive: in single-threaded applications, long running event handlers can prevent the processing of other events. Dein Name – Dein Vortragstitel Pra 01 - S. 5
  • 6. Advanced Developers Conference 10.-11. Oktober 2004 Bemerkungen Reactor: Example TAO: The high-performance, real-time CORBA ORB TAO, The ACE ORB, uses a multi-threaded Reactor design as its central server-side ORB core event handling infrastructure. Bemerkungen Reactor Reactor: Known Uses Object Request Brokers: The ACE ORB (TAO) provides Reactor-based event handling infrastructures. GUI infrastructures: MFC and Win Forms provide Reactor-based main Bemerkungen event loops. ACE: The open source ADAPTIVE Communication Environment provides several reusable Reactor frameworks for different types of event demultiplexers, such as WaitForMultipleObjects() and select(), and concurrency models, such as Leader/Followers. Dein Name – Dein Vortragstitel Pra 01 - S. 6
  • 7. Advanced Developers Conference 10.-11. Oktober 2004 Connection Establishment Bemerkungen (I) The two key activities performed by a connect connect web server are accepting client connection requests and processing connect subsequent HTTP requests. connect To provide a high throughput and quality of service to clients, processing a particular HTTP request must not prevent the web connect server from accepting new client connect connection requests. To support the configuration of the web get server towards different environments and get usage scenarios, it should be possible to modify connection establishment code independently of HTTP request processing Bemerkungen code. Connection Establishment (II) Separate the establishment of connections from processing events via Acceptor- Connector: dispatches * Event_ Reactor Acceptors (HTTP_Acceptor) Bemerkungen Handler maintains owns listen for incoming client * * connection request and select() Handle notifies create/initialize service handlers to process subsequent HTTP HTTP_ HTTP_ requests. Acceptor Handler Service handlers (HTTP_Handler) are created by an acceptor to process a specific client‘s HTTP requests. Dein Name – Dein Vortragstitel Pra 01 - S. 7
  • 8. Advanced Developers Conference 10.-11. Oktober 2004 Acceptor-Connector: Bemerkungen Problem Acceptor-Connector (POSA2) Clients Problem: In networked systems, components must first connect to one another before they can interact. However: 2: Send 1: Establish Logging Connection in many systems components can act both as Record This is a clients that initiate a connection to a remote logging record component or as servers that accept connection requests from remote clients. the code for connection establishment is largely Logging Server independent of service processing code and can also alter independently. if connection establishment code is interwoven with service processing code, components cannot handle new connection requests while they are processing service requests. Bemerkungen Output Media Acceptor-Connector: Structure Solution Structure: Separate Connector Service Handler Acceptor connection establishment from connect() handle_event() accept() service processing. Service Handlers implement the components that want to interact. Concrete Concrete Concrete Connector Service Handler Acceptor Bemerkungen Connectors actively establish new connections on behalf of a Service Handler to a remote Acceptor. activates creates and/or activates Acceptors passively accept connection requests from remote void connect(Service_Handler *h) { void accept() { Connectors and initialize a Service // Establish connection for // Create new Service Handler. // Service Handler h. Service_Handler *h = new …; Handler to service the connection. if (IPC_mechanism.connect(*h)) // Activate Handler, e.g., // Establish a connection // to register // between the client and Note: both synchronous and //with a Reactor. // the newly created activate(h) // Service Handler. asynchronous connection else IPC_mechanism.accept(h); // Error handling. establishment can be supported. } // Activate Handler, e.g., to // register with a Reactor. activate(h); } Dein Name – Dein Vortragstitel Pra 01 - S. 8
  • 9. Advanced Developers Conference 10.-11. Oktober 2004 Acceptor-Connector: Bemerkungen Dynamics Solution Dynamics: Let Connector and Acceptor establish a connection Handler 1 Acceptor Connector and pass them on to interacting Service Handlers. A Concrete Service Handler asks a connect(this) Concrete Connector to establish a accept connection to a specific remote Handler 2 activate Concrete Service Handler. The Connector sends a connection interact request to a Concrete Acceptor residing on the respective server. Connector and Acceptor establish the connection and pass them on to ‘their’ Concrete Service Handlers, which then interact. Bemerkungen Acceptor-Connector: Consequences Benefits Separation of concerns: connection establishment is separated from application-specific service processing. Reusability: connectors and acceptors are reusable Bemerkungen components. Efficiency: while service handlers still interact, connectors and acceptors can already establish a new connection. Liabilities Complexity: complex infrastructure and complex implementation for the asynchronous connection establishment. Dein Name – Dein Vortragstitel Pra 01 - S. 9
  • 10. Advanced Developers Conference 10.-11. Oktober 2004 Acceptor-Connector: Bemerkungen Example Call center applications, such as the Ericsson EOS Call Center Management System or Siemens FlexRouting, use Acceptor-Connector configurations to establish connections actively with passive supervisors (or agents). Public Network Bemerkungen Acceptor-Connector: Known Uses Web Browsers: Internet Explorer, Netscape to connect to servers associated with images embedded in html pages. Object Request Brokers: The ACE ORB (TAO) and Orbix 2000 to establish connections to ORB services. Bemerkungen ACE: provides a reusable Acceptor-Connector framework Call Center Management Systems: Ericsson, Siemens to establish connections to supervisors. Dein Name – Dein Vortragstitel Pra 01 - S. 10
  • 11. Advanced Developers Conference 10.-11. Oktober 2004 Bemerkungen Concurrency (I) A web server must serve multiple clients simultaneously and at a high- quality of service. Multi-threaded, concurrent event processing is a typical response to this requirement. Yet: Thread management overhead should be minimized to not degrade performance unnecessarily. Resource consumption should be predictable to avoid server saturation. Throughput is important. All events are therefore of identical priority, they are processed in the order of their arrival.Bemerkungen Concurrency (II) Let multiple Leader/Followers service threads share a common set of event sources by taking turns discovering demultiplexing, dispatching, and processing Thread_ shares Pool coordinates events: Bemerkungen (de)activates A thread pool (Thread_Pool) * Event_ coordinates a group of event handler Reactor Handler threads to share a common set of event maintains owns * * sources to process events. * select() Handle Event handlers (HTTP_Handler, notifies HTTP_Acceptor) join the thread pool to process client HTTP requests. HTTP_ HTTP_ Acceptor Handler A Reactor (Reactor) maintains the set of shared event sources. Handles (Handle) represent the set of event sources shared by the event handler threads. Dein Name – Dein Vortragstitel Pra 01 - S. 11
  • 12. Advanced Developers Conference 10.-11. Oktober 2004 Leader/Followers: Bemerkungen Problem Leader/Followers (POSA2) Clients Problem: An event-driven application may use multi-threading to serve multiple client requests simultaneously. However: Concurrency overhead, such as thread-creation, context switching, thread coordination, should not outweigh the benefits of multi-threading. Resource consumption should be predictable to avoid saturation of the event-driven Logging Server application. Some concurrent application also trade general throughput and performance over scheduling and prioritizing. Requests are structured, repetitive, require immediate response, and can be served via Bemerkungen short, atomic, and isolated processing actions. Output Media Leader/Followers: Structure Solution Structure: Introduce a thread demultiplexes Thread Pool Event Handler * pool in which multiple service instances join() handle_event() share a common event source. uses leader() Events arrive on a Handle Set shared by Concrete all threads. Event Handler Handle Set Bemerkungen Event Handler threads implement application services: follower threads • A Leader Thread gets exclusive access to processing Event threads the Handle Set, and blocks until an event Handler Event arrives. Handler • Follower Threads queue up behind the Event Event leader and wait until it is their turn to be Handler Handler the leader. • Processing Threads are processing an Event Event Handler Handler event they received when being the leader. Handle Set A Thread Pool coordinates the Event leader thread events Handler threads. Dein Name – Dein Vortragstitel Pra 01 - S. 12
  • 13. Advanced Developers Conference 10.-11. Oktober 2004 Leader/Followers: Bemerkungen Dynamics Solution Dynamics: Let the Event follower threads Handler threads take turns on the shared Handle Set. processing Event threads All threads are spawned. One thread obtains Handler Event access to the Handle Set and becomes the 6 Handler Leader thread (1), all other threads become Followers. An event occurs on the Handle Set and is Event Event received by the Leader thread (2). Handler Handler promote The Leader thread changes its role to a leader Processing thread and processes the request 5 (3), the Thread Pool promotes one Follower 4 Event Event to become the new Leader (4). Handler Handler While the event is processed (5), steps (2) - 3 1 (4) recur. Handle When the event processing finished the Set 2 Processing thread changes its role to a Bemerkungen leader Follower and waits until it becomes the new thread events Leader again (6). Leader/Followers: Consequences Benefits Simple thread management: threads coordinate themselves via the pool; all threads are spawned at system start-up. Bemerkungen Predictable resource consumption: the thread pool is of fixed size Performance: One of the fastest—if not the fastest— concurrency architecture. Liabilities Most suitable for applications with a small set of atomic services. Only suitable for systems whose services have a small memory footprint. No scheduling / priority handling possible. Dein Name – Dein Vortragstitel Pra 01 - S. 13
  • 14. Advanced Developers Conference 10.-11. Oktober 2004 Leader/Followers: Bemerkungen Example On-Line Transaction Processing (OLTP) Systems, such as for travel agencies, financial services, or broker information, often use the a Leader/Followers concurrency architecture to perform high- volume transaction processing on backend database servers. Bemerkungen Leader/Followers: Known Uses Object Request Brokers: The ACE ORB (TAO) and to handle network events. Application Servers: Application Server to handle network events. Bemerkungen Transaction Monitors: Tuxedo, MTS Transaction Service to process transactions. TAO Features • Open-source • 500+ classes • 500,000+ lines of C++ • ACE/patterns-based • 30+ person-years of effort • Ported to UNIX, Win32, MVS, and many RT & embedded OSs (e.g., VxWorks, LynxOS, Chorus, QNX) Dein Name – Dein Vortragstitel Pra 01 - S. 14
  • 15. Advanced Developers Conference 10.-11. Oktober 2004 Concurrency Coordination Bemerkungen (I) To avoid concurrency hazards like race conditions, the access sequence of HTTP Processing Threads threads to shared web server HTTP_ HTTP_ components must be coordinated. HTTP_ HTTP_ Handler Handler HTTP_ HTTP_ Handler Handler Handler Handler Some of these shared components—such as the Thread_Pool—are relatively small-sized Thread_Pool objects with a fairly simple, straight-forward logic. Reactor It is crucial that the Thread_Pool executes efficiently—it is the heart of the web servers The thread pool Is shared by the event handling infrastructure. Therefore, the HTTP_Handlers thread coordination mechanism should be To get access to The web server‘s efficient. Event sources. Expensive thread coordination overhead Bemerkungen should be avoided. Concurrency Coordination (II) Coordinate the access to the Thread_Pool by implementing it as Thread_ a Monitor Object: shares Pool dispatches join() The thread pool (Thread_Pool) is a leader() * Bemerkungen monitor object. (de) Event_ Reactor activates The thread pool‘s services (join(), Handler maintains owns Thread_Pool::join() { leader()) are synchronized methods. // Acquire mutex lock automatically. * * Guard<Thread_Mutex> guard (mutex_); * select() Handle The tread pool implementation uses a // run the leader, follower, processor loop. notifies for (;;) { // for ever ... // We have a leader, so sleep ... monitor lock (mutex_) and a condition while (leader_thread != NO_LEADER) condition_.wait(timeout); variable (condition_) to synchronize HTTP_ HTTP_ // Allow other threads to join the pool. Acceptor Handler guard.release(); the access to its services transparently // We are the leader, so access the // event sources. for the HTTP processing threads reactor_->handle_events(); (HTTP_Acceptor , HTTP_Handler). // Reenter monitor to serialize the test. guard.acquire(); } } Dein Name – Dein Vortragstitel Pra 01 - S. 15
  • 16. Advanced Developers Conference 10.-11. Oktober 2004 Bemerkungen Monitor Object: Problem Monitor Object (POSA2) Problem: Concurrent access to ‘small’ A Call Processing concurrent objects such as a queues or System stacks should be as simple as possible: Synchronization should be easy to implement. +49/89/909090 Threading and synchronization overhead should be minimized or, even better, +49/89/232323 A Call avoided. Processing +49/89/111222 Queue Yet clients should not notice that the object +49/89/999999 is shared. +49/89/987654 +49/89/303030 Bemerkungen Monitor Object: Structure Solution Structure: Synchronize Monitor Object an object’s methods so that only sync_method1() one method can execute at any sync_method2() one time: uses uses 1..* A Monitor Object is an object Monitor Lock Monitor Cond. Bemerkungen multiple threads. shared among acquire() wait() release() notify() Synchronized Methods implement thread-safe functions of the Monitor Object. void sync_method1() { lock.acquire(); // Acquire monitor lock. One or more Monitor Conditions void sync_method2() { ... // Start processing. lock.acquire(); allow the Synchronized Methods // Suspend thread on a monitor condition ... // Start processing. // if immediate progress is impossible. to schedule their execution. if (progress_impossible) if (progress_impossible) condition_1.wait(); condition_2.wait(); A Monitor Lock ensures a ... // Resume processing. ... // Resume processing. serialized access to a Monitor // Notify waiting threads that can condition_1.notify(); Object. // potentially resume their processing // because we are done. lock.release(); condition_2.notify(); } lock.release(); // Release monitor lock. } Dein Name – Dein Vortragstitel Pra 01 - S. 16
  • 17. Advanced Developers Conference 10.-11. Oktober 2004 Bemerkungen Monitor Object: Dynamics Solution Dynamics: Let the Monitor Object schedule its own method Client Client Monitor Monitor Monitor execution sequence. Thread1 Thread2 Object Lock Cond. A Client Thread invokes a Synchronized method acquire Method. The method acquires the Monitor Lock and starts executing. wait automatic At some point, the method cannot proceed, automatic release suspend therefore it suspends itself on the Monitor Condition, which automatically releases the acquire method lock and puts the calling thread to sleep. Another thread can execute a Synchronized notify Method on the monitor object. Before terminating, this method notifies the Monitor Condition on which the first thread is waiting. release If the Monitor Condition evaluates to true, it automatic resumption automatic automatically re-acquires the Monitor Lock acquisition and resumes the suspended thread as well as release the suspended Synchronized Method. Bemerkungen Monitor Object: Consequences Benefits Simple yet effective: Simplification of concurrency control; simplification of scheduling method execution. Bemerkungen Liabilities Limited scalability: clients can block. Hard to extend: functionality and synchronization logic are tightly coupled. Nested Monitor Lockout Problem. Dein Name – Dein Vortragstitel Pra 01 - S. 17
  • 18. Advanced Developers Conference 10.-11. Oktober 2004 Bemerkungen Monitor Object: Example class Message_Queue { Message queues are often public: enum { MAX_MESSAGES = /* ... */; }; implemented as Monitor Message_Queue (size_t max_messages = MAX_MESSAGES); /* synchronized */ void put (const Message &msg); /* synchronized */ Message get (); Objects to ensure their thread- /* synchronized */ bool empty () const; /* synchronized */ bool full () const; safe operation. private: void put_i (const Message &msg); Message get_i (); bool empty_i () const; bool full_i () const; bool Message_Queue::empty () const { Guard<Thread_Mutex> guard (monitor_lock_); size_t message_count_; return empty_i (); size_t max_messages_; } mutable Thread_Mutex monitor_lock_; Thread_Condition not_empty_; bool Message_Queue::full () const { Thread_Condition not_full_; Guard<Thread_Mutex> guard (monitor_lock_); } return full_i (); } void Message_Queue::put (const Message &msg) { Guard<Thread_Mutex> guard (monitor_lock_); while (full_i ()) not_full.wait (); put_i (msg); not_empty.notify (); } Message Message_Queue::get () { Guard<Thread_Mutex> guard (monitor_lock_); while (empty_i ()) not_empty.wait (); Bemerkungen Message m = get_i (); not_full.notify (); return m; } Monitor Object: Known Uses ADAPTIVE Communication Environment: ACE provides a communication gateway application that uses Monitor Objects to improve performance on multi-processors. Bemerkungen .NET, Java: the main synchronization mechanism in .NET and Java is based on Monitors. Dein Name – Dein Vortragstitel Pra 01 - S. 18
  • 19. Advanced Developers Conference 10.-11. Oktober 2004 Efficient Content Access Bemerkungen (I) To achieve a high throughput and a good quality of service, a web server must access URLs requested by clients efficiently. Retrieving the URLs from the file system for every access to them introduces performance penalties, however: Some URLs are more frequently accessed than others. Individual clients often return to an URL they already visited before. Bemerkungen Efficient Content Access (II) Virtual_ Introduce a virtual file system to Filesystem have the content of frequently accessed URLs readily accessible via Caching: Thread_ A cache (Virtual_Filesystem) Bemerkungen shares Pool dispatches join() keeps URL content in memory once leader() * it got accessed and loaded from the (de) Event_ Reactor activates file system for the first time. For Handler maintains owns subsequent accesses, this content is * * * readily available select() Handle notifies HTTP_Handlers first check the HTTP_ HTTP_ cache for URL availability before Acceptor Handler they load it from the file system. Dein Name – Dein Vortragstitel Pra 01 - S. 19
  • 20. Advanced Developers Conference 10.-11. Oktober 2004 Bemerkungen Caching: Problem Caching (POSA3) Clients Problem: Repetitious acquisition, initialization, 2: Send Logging 1: Establish and release of resources Record Connection 4: Send Logging causes unnecessary 3: Establish This is a logging Record record Connection performance overhead: 6: Send Logging 5: Establish Record Connection The costs of acquisition, Expensive, initialization, and release of Logging Server if we must create the frequently used resources connection should be minimized. each time! Access to the resources should be simple. Bemerkungen Output Media Caching: Structure Solution Structure: Instead of releasing a resource after its usage, temporarily store it in an in-memory buffer: Cache Client Resources represent entities that Bemerkungen acquire() application functionality uses to release() execute its business logic, such as maintains threads, network connections, file * handles, memory, etc. Resource A Cache keeps currently unused resources of a specific type readily available for subsequent use. Clients use the Resources maintained by the Cache to execute their own functionality. Dein Name – Dein Vortragstitel Pra 01 - S. 20
  • 21. Advanced Developers Conference 10.-11. Oktober 2004 Bemerkungen Caching: Dynamics Solution Dynamics: Create resources once and re-cycle them often: Client Cache A Client requests a Resource from the Cache. create acquire The Cache checks if the Resource is available Resource and creates one, if not Resource The Client uses the Resource. use The Client releases the resource by passing it back to the Cache. release Resource The Cache keeps the Resources for subsequent uses. The Client requests the Resource from the acquire Cache again. Resource The Cache returns the already existing use Resource. The Client ‚re-uses‘ the Resource. Bemerkungen Caching: Consequences Benefits • Performance: Quick access to resources, minimized resource acquisition and release costs. • Predictability: Explicit control of resource usage possible. Bemerkungen Liabilities • Cache maintenance overhead: If the cache is full, its ‚re- organization‘ can cause performance penalties in accessing and using resources. Dein Name – Dein Vortragstitel Pra 01 - S. 21
  • 22. Advanced Developers Conference 10.-11. Oktober 2004 Bemerkungen Caching: Example Web Browsers: Web Browsers like Netscape and Internet Explorer use a cache to keep local copies of frequently accessed documents. Bemerkungen Caching: Known Uses Web Browsers: Internet Explorer and Netscape cache frequently accessed documents. Web Servers: Apache and Jaws cache frequently requested Bemerkungen documents. Object Request Brokers: The ACE ORB (TAO) caches unused network connections and re-cycles them for new client connections. High-performance applications: Almost every high-performance software uses caches to speed-up performance. Dein Name – Dein Vortragstitel Pra 01 - S. 22
  • 23. Advanced Developers Conference 10.-11. Oktober 2004 Bemerkungen One Cache Only (I) To achieve predictability regarding resource Virtual_ One file Filesystem consumption, there should system only, please! only be one virtual file system in the web server. Thread_ Pool shares dispatches join() However, OO languages do not leader() * Event_ support an instance-controlled (de) Reactor Handler activates object creation. maintains owns * * * select() Handle notifies HTTP_ HTTP_ Acceptor Handler Bemerkungen One Cache Only (II) Virtual_ Ensure that the web server‘s Filesystem Virtual_Filesystem can be class Virtual_Filesystem { static Virtual_Filesystem *theInstance; instantiated just once by making Virtual_Filesystem () { /* ... */ }; it a Singleton: public: static Virtual_Filesystem *instance() { Thread_ // Create file system, if necessary. if (! theInstance) Bemerkungen A singleton shares Pool dispatches theInstance = new Virtual_Filesystem ; // Return the unique file system instance join() (Virtual_Filesystem) keeps its return theInstance; } leader() } * constructors and destructors private (de) Event_ // Static initialization of the singleton. Reactor activates Virtual_Filesystem * so that only itself can create Handler Virtual_Filesystem ::theInstance = 0; maintains owns instances of itself. * * * select() Handle notifies A class access method ensures that only one instance of a singleton class HTTP_ HTTP_ can be created and offers clients an Acceptor Handler access to this unique instance. Note that we trade safety for flexibility here! Dein Name – Dein Vortragstitel Pra 01 - S. 23
  • 24. Advanced Developers Conference 10.-11. Oktober 2004 Bemerkungen Cache Management (I) If we only load URLs into the web server‘s cache, it will be full at some point in time and we cannot load any further URLs. The less frequently the loaded URLs are accessed, the more the performance and throughput of the web server degrades. Any run-time flushing of the cache must not degrade the operational performance of the web server. Virtual_Filesystem Bemerkungen Cache Management (II) Manage the content of the web Virtual_ Evictor Filesystem server‘s Virtual_Filesystem and release unused URLs via an Evictor: Thread_ An evictor (Evictor) runs in its Bemerkungen shares Pool dispatches own thread of control. Once the join() leader() occupation of the * (de) Event_ Virtual_Filesystem exceeds a Reactor activates Handler certain limit, it notifies the evictor to maintains owns * * remove less frequently used URLs * select() Handle from it. notifies HTTP_ HTTP_ Acceptor Handler Dein Name – Dein Vortragstitel Pra 01 - S. 24
  • 25. Advanced Developers Conference 10.-11. Oktober 2004 Bemerkungen Evictor: Problem Evictor (POSA3) Problem: Distributed systems need to manage resources such as file handles or CPU time efficiently. Over time resources that are not User required any more should be freed. Immediate resource Resource release, however, might cause create ? expensive re-acquisitions: The frequency of resource usage should influence its lifecycle. use Resource release should be parametrized by issues such as X CPU time. release The solution should be transparent to the user. Bemerkungen Evictor: Structure Solution Structure: Monitor resources and control their lifecycles using a strategy such as LRU. Each resource is marked on usage. Recently or Resouce User infrequently used resources do not get marked. The application periodically or on demand selects Bemerkungen unmarked resources for eviction: Resource Resource users represent applications or OS that use resources. A Resource provides some local or Evictor remote service. An Evictor releases resources depending one or more eviction Eviction strategies. Strategy An Eviction Strategy describes the strategy used to determine whether a resource should be evicted. Dein Name – Dein Vortragstitel Pra 01 - S. 25
  • 26. Advanced Developers Conference 10.-11. Oktober 2004 Bemerkungen Evictor: Dynamics Solution Dynamics: Register resources Resource User Resource Evictor with the evictor. Evictor checks resources for usage marks. If strategy tells evictor access update to free resource evictor asks resource mark whether it is evictable. If evictable resource is informed and released. check mark eviction strategy Is Evictable? Yes! beforeEviction evict Bemerkungen Evictor: Consequences Benefits • Scalability: Evictor helps to keep upper bound of resources. Applications can scale without memory impact. • Low-memory footprint: Eviction strategy helps to keep only required resources in memory, thus reducing memory Bemerkungen footprint. • Transparency: User does not see implementation of evictor. • Stability: Reduces propability of resource exhaustion. Liabilities • Overhead: Additional logic to determine which resources to evict ad to implement eviction strategy. • Re-Acquisition Penalty: Resurrecting an already evicted resource might be expensive. Finetune eviction strategy to prevent this. Dein Name – Dein Vortragstitel Pra 01 - S. 26
  • 27. Advanced Developers Conference 10.-11. Oktober 2004 Bemerkungen Evictor: Known Uses .NET: Garbage Collection. EJB: Release of enterprise beans. Weak References in .NET and Java: Using weak references disposable objects can be used and resurrected. CORBA: Servant Manager uses eviction to get rid of servants. Operating Systems: Swapping implemenations use the Evictor pattern. Bemerkungen Flexible Cache Management (I) Different web server usage scenarios could require different cache eviction ? policies. Hard-coding cache eviction policies into Bemerkungen the evictor pollutes its code with many ? conditional statements, which ultimately impede performance. Virtual_Filesystem Hard-coding cache eviction policies into the evictor degrades its maintainability LRU vs. LFU vs. ?? and extensibility. Changing the current cache eviction policy must be possible at run-time to avoid shutting down the web-server (means taking it out of service) unnecessarily. Dein Name – Dein Vortragstitel Pra 01 - S. 27
  • 28. Advanced Developers Conference 10.-11. Oktober 2004 Flexible Cache Bemerkungen Management (II) Configure the Evictor with cache eviction policies dynamically via Virtual_ Evictor LFU Filesystem Strategies: Abstract_ An abstract strategy LRU Eviction (Abstract_Eviction) defines the interface for different cache eviction Thread_ shares Pool dispatches policies. join() leader() Concrete Strategies (LFU, LRU) implement * (de) Event_ the different cache eviction policies. Reactor activates Handler maintains owns A context (Evictor) is (dynamically) * * * configured with a particular concrete select() Handle notifies strategy. This strategy is called whenever the context wants to execute the HTTP_ HTTP_ Acceptor Handler respective type of algorithm. Bemerkungen Reliable Synchronization (I) Components of our web server that HTTP_ Evictor Handler are accessed by multiple threads concurrently must be synchronized to avoid race conditions. Protecting critical sections in shared Bemerkungen remove insert components, such as the modification of file cache content, prevents damaging the state of the web server. However: Locks that protect the critical sections in the shared object should always be class Virtual_Filesystem { public: acquired and released properly. int insert( ... ) { // Critical section. Virtual_Filesystem mutex_.acquire(); It is hard to ensure that locks are // Do something ... if error return; // Bad! released in all paths through the shared // Do something ... mutex_.release(); object’s code. } private: Thread_Mutex mutex_; }; Dein Name – Dein Vortragstitel Pra 01 - S. 28
  • 29. Advanced Developers Conference 10.-11. Oktober 2004 Reliable Synchronization Bemerkungen (II) Virtual_ Evictor LFU Protect critical regions in the class Virtual_Filesystem { Filesystem public: int insert( ... ) { Virtual_Filesystem via // Critical section. TM_Guard guard(mutex_); Abstract_ Scoped Locking: LRU TM_Guard // Do something ... Eviction if error return; // O.K.! // Do something ... A guard class (TM_Guard) } Thread_ private: Thread_Mutex mutex_; automates the acquisition and shares Pool dispatches }; class TM_Guard { join() release of a given type of public: leader() TM_Guard * (Thread_Mutex &m) { lock. (de) mutex_ = &l; Event_ activates mutex_->acquire(); Reactor } Handler The shared component maintains owns ~TM_Guard() { mutex_->release(); * * (Virtual_Filesystem) } * private: select() Handle Thread_Mutex *mutex_; uses the guard to protect its notifies }; critical section from HTTP_ HTTP_ concurrent access. Acceptor Handler Bemerkungen Scoped Locking: Problem Scoped Locking (POSA2) class Foo { Problem: Protecting critical ThreadMutex lock; sections from concurrent public: access proxies prevents void bar() { Bemerkungen damaging state in an object. // Do something ... However: // Acquire lock to serialize the // access to the critical section. Locks that protect the critical lock.acquire(); sections should always be D! // Do something ... BA acquired and released properly. if (error) return; It is hard to ensure that locks // Release lock. are released in all exit paths lock.release(); from the critical section. // Do something ... } Dein Name – Dein Vortragstitel Pra 01 - S. 29
  • 30. Advanced Developers Conference 10.-11. Oktober 2004 Bemerkungen Scoped Locking: Structure Solution Structure: Scope the critical class Foo { section and acquire and releases a lock ThreadMutex lock; automatically. public: void bar() { A Guard acquires a lock in its constructor // Do something ... and releases this lock in its destructor. { // Begin of critical section. A scoped Critical Section acquires a guard Guard guard(lock); object to serialize access to it. // Do something ... .! O. K if (error) class Guard { return; ThreadMutex *lock; } // End of critical section. public: Guard (ThreadMutex &l) { // Do something ... lock = &l; } lock->acquire(); } Guard() { Bemerkungen lock->release(); } } Scoped Locking: Dynamics class Foo { Solution Dynamics: Acquire a lock explicitly ThreadMutex lock; and let the run-time environment release public: void bar() { it properly: // Do something ... { // Begin of critical section. Creating the Guard instance on the stack 1a Guard guard(lock); after entering the scope of the Critical Bemerkungen // Do something ... Section acquires the lock. if (error) return; Leaving the Critical Section means leaving 2a } // End of critical section. its scope, which lets the run-time // Do something ... } environment call the destructor of the Guard on the stack, which releases the class Guard { ThreadMutex *lock; lock. public: 1b Guard (ThreadMutex &l) { lock = &l; lock->acquire(); } 2b ~Guard() { lock->release(); } } Dein Name – Dein Vortragstitel Pra 01 - S. 30
  • 31. Advanced Developers Conference 10.-11. Oktober 2004 Scoped Locking: Bemerkungen Consequences Benefits Robustness: Locks are acquired and released properly— regardless of the exit path from the critical section. In Java: Built-in language feature. Liabilities For example, in C++ Language Feature Dependent: Works only “inside” true C++. For instance, calling the C function longjmp() does not call destructors of C++ objects when Bemerkungen the stack unwinds. Scoped Locking: Example .NET defines a feature called a synchronized block that implements Scoped Locking. Bemerkungen class Dispatcher { ... public void changeHandler(Handler newHandler) { lock(eventSource) { oldHandler = eventSource.getHandler(event); eventSource.installHandler(event, newHandler); } } ... private EventSource eventSource; private Event event; private Handler oldHandler; } Dein Name – Dein Vortragstitel Pra 01 - S. 31
  • 32. Advanced Developers Conference 10.-11. Oktober 2004 Scoped Locking: Known Bemerkungen Uses Concurrent software: Almost all concurrent software systems use scoped locking to ensure a proper acquisition and release of locks that protect critical regions. Class libraries: Class libraries like ACE, threads.h++, and the Booch components define a set of guard classes that follow scoped locking. .NET, Java: the synchronized blocks provide built-in scoped locking. Bemerkungen Efficient Synchronization (I) A web server should be able to run HTTP_ Evictor Handler on multiple platforms, ranging from single to multi-processor platforms. On different platform, different types of Bemerkungen remove insert lock provide different performance. For instance, a mutex is most suitable for single processor platforms whereas a readers/writer lock is often class RW_Virtual_Filesystem { public: most efficient for multi-processor int insert( ... ) { // Critical section. class TM_Virtual_Filesystem { RW_Guard guard(rw_); platforms. public: // Do something ... int insert( ... ) { if error // Critical section. return; Virtual_Filesystem TM_Guard guard(mutex_); // Do something ... However, maintaining multiple variants // Do something ... } e if error abl private: of every shared object that differ only return; Readers_Writer rw_; tain // Do something ... }; } in in the lock type is very ineffective. private: ma Thread_Mutex mutex_; Un }; Dein Name – Dein Vortragstitel Pra 01 - S. 32
  • 33. Advanced Developers Conference 10.-11. Oktober 2004 Efficient Synchronization Bemerkungen (II) Virtual_ Evictor LFU Configure the type of lock used by template <class LOCK> Filesystem class Virtual_Filesystem { public: the Virtual_Filesystem via int insert( ... ) { // Critical section. Abstract_ Strategized Locking: LRU Guard Guard<Lock> guard(lock_); Eviction // Do something ... if error return; A guard class (Guard) automates // Do something ... Thread_ } private: the acquisition and release of a LOCK lock_; shares Pool dispatches }; join() template <class LOCK> configurable type of lock. class Guard { leader() public: * Guard(LOCK &l) { (de) Event_ lock_ = &l; The shared component Reactor activates lock_.acquire(); Handler } (Virtual_Filesystem) uses the ~Guard() { maintains owns lock_.release(); * * } guard to protect its critical section private: * LOCK *lock_; select() Handle from concurrent access and }; notifies configures it with the appropriate HTTP_ HTTP_ type of lock. Acceptor Handler Bemerkungen Strategized Locking: Problem Strategized Locking (POSA2) class Singleton { static Singleton *theInstance; Singleton () { /* ... */ }; public: static Singleton *getInstance() { Problem: The most optimal locking if (! theInstance) { // Acquire lock to serialize the mechanisms for a piece of code can // access to the critical section. lock.acquire(); vary dependent on the platform // Create singleton, if necessary. if (! theInstance) Bemerkungen theInstance = new Singleton; used. However: // Release lock. lock.release(); } Maintaining multiple variants of code // Return the singleton instance. return theInstance; that differs only in the lock type } } used is very ineffective. // Static singleton initialization. Singleton *Singleton::theInstance = 0; Thread mutex on single-processor platforms. Readers/Writer lock on multi-processor platforms. Dein Name – Dein Vortragstitel Pra 01 - S. 33
  • 34. Advanced Developers Conference 10.-11. Oktober 2004 Strategized Locking: Bemerkungen Structure Solution Structure: Modify template <class LOCK> class Singleton { static Singleton *theInstance; the code so that it can be Singleton () { /* ... */ }; configured with the LOCK lock; appropriate type of lock, public: either statically using static Singleton *getInstance() { if (! theInstance) { generic types or // Acquire lock to serialize the dynamically using // access to the critical section. lock.acquire(); polymorphism. // Create singleton, if necessary. if (! theInstance) theInstance = new Singleton; // Release lock. lock.release(); } // Return the singleton instance. return theInstance; Bemerkungen } } Strategized Locking: Dynamics Solution Dynamics: template <class LOCK> class Singleton { static Singleton *theInstance; Configure the code with Singleton () { /* ... */ }; the appropriate type of LOCK lock; lock during its public: initialization. static Singleton *getInstance() { Bemerkungen if (! theInstance) { // Acquire lock to serialize the // access to the critical section. lock.acquire(); // Create singleton, if necessary. if (! theInstance) theInstance = new Singleton; // Release lock. lock.release(); } // Return the singleton instance. return theInstance; } } Singleton<ThreadMutex> *Singleton<ThreadMutex>::theInstance = 0; Dein Name – Dein Vortragstitel Pra 01 - S. 34
  • 35. Advanced Developers Conference 10.-11. Oktober 2004 Strategized Locking: Bemerkungen Consequences Benefits Flexibility: Critical sections can be configured with the most appropriate type of lock, dependent on usage scenarios and platforms on which the code runs. Maintenance: Code becomes easier to maintain and (re-)use because it becomes independent on locking mechanisms. Liabilities Visible Locking: If templates are used it is visible to application code what locking mechanism is used in a piece of code. Bemerkungen Strategized Locking: Example ACE: The open source ADAPTIVE Communication Environment provides a reusable lock guard class that uses templatized strategized locking. Bemerkungen template <class LOCK> class Guard { public: Guard(LOCK &l) : lock_ (&lock), owner_ (false) { lock_.acquire(); owner_ = true; } ~Guard() { if (owner_) lock_.release(); } private: LOCK *lock_; bool owner_ }; Dein Name – Dein Vortragstitel Pra 01 - S. 35
  • 36. Advanced Developers Conference 10.-11. Oktober 2004 Strategized Locking: Bemerkungen Known Uses Concurrent software: Almost all concurrent software systems use strategized locking to ensure a proper acquisition and release of locks that protect critical regions. Class libraries: Class libraries like ACE use strategized locking. Bemerkungen Efficient Synchronization (III) The web server‘s virtual file HTTP_ Evictor Handler system is Singleton. It is also a component shared by multiple threads, thus we must prevent an accidental double Bemerkungen remove insert instantiation, due to race conditions. However, all locking and template <class LOCK> class Virtual_Filesystem { synchronization overhead should static Virtual_Filesystem *theInstance; Virtual_Filesystem () { /* ... */ }; be minimized. public: static Virtual_Filesystem *instance() { // Create file system, if necessary. Guard<LOCK> guard(lock_); if (! theInstance) Virtual_Filesystem theInstance = new Virtual_Filesystem ; // Return the unique file system instance return theInstance; } } ! w!! // Static initialization of the singleton. Slo Virtual_Filesystem * Virtual_Filesystem ::theInstance = 0; Dein Name – Dein Vortragstitel Pra 01 - S. 36
  • 37. Advanced Developers Conference 10.-11. Oktober 2004 Efficient Synchronization Bemerkungen (IV) Virtual_ Evictor LFU Introduce a flag that provides Filesystem a hint whether it is Abstract_ necessary to execute a LRU Guard template <class LOCK> Eviction class Virtual_Filesystem { critical section before static Virtual_Filesystem *theInstance; Virtual_Filesystem () { /* ... */ }; Thread_ acquiring the lock that public: staticshares Pool dispatches Virtual_Filesystem *instance() { guards it, using Double- // Virtual file system already there?? join() if (! theInstance) { // First Check!! // Create file system, if necessary. leader() Checked Locking. Guard<LOCK> guard(lock_); * if (! theInstance) // Double Check!! (de) Event_ theInstance = new Virtual_Filesystem; Reactor activates } Handler // Return the unique file system instance return theInstance; maintains owns } * * } select() notifies * // Static initialization of the singleton. Handle Virtual_Filesystem * Virtual_Filesystem ::theInstance = 0; HTTP_ HTTP_ Acceptor Handler Bemerkungen Double-Checked Locking: Problem Double-Checked Locking class Singleton { static Singleton *theInstance; (POSA2) Singleton () { /* ... */ }; public: Problem: Sometimes, critical static Singleton *getInstance() { // Acquire lock to serialize the sections are not always Bemerkungen // access to the critical section. executed, but only conditionally. lock.acquire(); Some such critical sections are // Create singleton, if necessary. even executed just once. W! if (! theInstance) LO theInstance = new Singleton; However: S // Release lock. Despite the need for serializing lock.release(); the access to the critical section, // Return the singleton instance. there should be no locking return theInstance; overhead for those situations in } } which the critical section is not executed. // Static singleton initialization. Singleton *Singleton::theInstance = 0; Dein Name – Dein Vortragstitel Pra 01 - S. 37
  • 38. Advanced Developers Conference 10.-11. Oktober 2004 Double-Checked Locking: Bemerkungen Structure class Singleton { Solution Structure: static Singleton *theInstance; Double-Check: Singleton () { /* ... */ }; Introduce a flag that public: static Singleton *getInstance() { provides a hint whether if (! theInstance) { 1st Check it is necessary to // Acquire lock to serialize the // access to the critical section. execute a critical lock.acquire(); section before acquiring // Create singleton, if necessary. the lock that guards it. if (! theInstance) 2nd Check theInstance = new Singleton; // Release lock. lock.release(); } // Return the singleton instance. return theInstance; } } Bemerkungen // Static singleton initialization. Singleton *Singleton::theInstance = 0; Double-Checked Locking: Dynamics class Singleton { Solution Dynamics: Skip the critical static Singleton *theInstance; section if it does not need to be Singleton () { /* ... */ }; executed: public: static Singleton *getInstance() { If the first check reveals that the 1 if (! theInstance) { Bemerkungen critical section can be skipped. No // Acquire lock to serialize the // access to the critical section. locking overhead occurs. 2 lock.acquire(); If the critical section needs to be // Create singleton, if necessary. executed, a lock is acquired to 3 if (! theInstance) theInstance = new Singleton; prevent race conditions if two or more threads passed the first // Release lock. lock.release(); check. } // Return the singleton instance. Once the lock is acquired, the return theInstance; need for executing the critical } } section is checked again, so that it is executed only when needed. // Static singleton initialization. Singleton *Singleton::theInstance = 0; Dein Name – Dein Vortragstitel Pra 01 - S. 38
  • 39. Advanced Developers Conference 10.-11. Oktober 2004 Double-Checked Locking: Bemerkungen Consequences Benefits Avoids locking overhead: Locks are acquired only when necessary, the most common cases perform fast. Liabilities Awkward implementation: Can require processor-specific instructions on certain platforms due to multi-processor cache coherency. Does not work in Java! Bemerkungen Double-Checked Locking: Example Solaris: The implementation of errno on Solaris uses double-checked locking. #define errno (*(___errno()) int *___errno () { static pthread_mutex_t keylock; Bemerkungen static pthread_key_t key; static int once; int *error_number = 0; if (once) { pthread_mutex_lock (&keylock); if (once) { pthread_key_create (&key, free); once = 1; } pthread_mutex_unlock (&keylock); } pthread_getspecific (key, &error_number); if (error_number == 0) { error_number = (int *) malloc (sizeof (int)); pthread_setspecific (key, error_number); } return error_number; } Dein Name – Dein Vortragstitel Pra 01 - S. 39
  • 40. Advanced Developers Conference 10.-11. Oktober 2004 Double-Checked Locking: Bemerkungen Known Uses ADAPTIVE Communication Environment (ACE): ACE uses double-checked locking throughout its implementation. Operating Systems: Solaris and Linux use double-checked locking, for instance to implement POSIX once variables or POSIX Pthreads. Bemerkungen Accessing OS APIs (I) // Handle UNIX/Win32 portability. Developing portable web server is #if defined (_WIN32) hard if developers must wrestle static CRITICAL_SECTION lock; with low-level operating system #else static mutex_t lock; APIs. #endif /* _WIN32 */ Bemerkungen(_WIN32) OS APIs differ from platform to ... #if defined platform regarding both syntax and SOCKET h; semantic. DWORD t_id; #else OS dependencies in system code int h; result in non-portable code. thread_t t_id; There is a mismatch between #endif /* _WIN32 */ ... procedural programming style of OS #if defined (_WIN32) APIs and the object paradigm used ThreadCreate( ... ); in the component’s implementation. #else thr_create( ... ); Programming low-level APIs is #endif /* _WIN32 */ inherently error-prone. Dein Name – Dein Vortragstitel Pra 01 - S. 40
  • 41. Advanced Developers Conference 10.-11. Oktober 2004 Bemerkungen Accessing OS APIs (II) Virtual_ Evictor LFU Shield the web server Filesystem implementation from low-level Abstract_ OS-APIs via a set of Wrapper LRU Guard Eviction Facades: Thread_ A set of wrapper facades (Socket, shares Pool dispatches join() Thread, Thread_Mutex, leader() * Condition_Variable) provide a (de) Event_ Reactor activates portable, robust, and modular Handler maintains owns access to low-level OS APIs. * * * select() Handle Application classes (our web server notifies classes) use the wrapper facades in HTTP_ HTTP_ their implementation. Acceptor Handler Thread_ Condition_ Bemerkungen Socket Thread Mutex Variable Wrapper Facade: Problem Wrapper Facade (POSA2) // Main driver function for a logging server. int main (int argc, char *argv[]) { struct sockaddr_in sock_addr; // Handle UNIX/Win32 portability. #if defined (_WIN32) Problem: Many low-level APIs SOCKET acceptor; WORD version_requested = MAKEWORD(2, 0); differ in syntax, even if they WSADATA wsa_data; int error = WSAStartup(version_requested, &wsa_data); if (error != 0) return -1; share semantics. Yet: #else int acceptor; Bemerkungen #endif /* _WIN32 */ applications should be portable. // Create a local endpoint of communication. acceptor = socket (PF_INET, SOCK_STREAM, 0); direct dependencies of // Set up the address to become a server. memset (&sock_addr, 0, sizeof sock_addr); application code to these APIs sock_addr.sin_family = PF_INET; sock_addr.sin_port = htons (LOGGING_PORT); should be avoided. sock_addr.sin_addr.s_addr = htonl (INADDR_ANY); // Associate address with endpoint. applications should use these bind (acceptor, reinterpret_cast<struct sockaddr *> (&sock_addr), sizeof sock_addr); APIs correctly and consistently. // Make endpoint listen for connections. listen (acceptor, 5); // Block waiting for clients to connect. h = accept (acceptor, 0, 0); // Main server event loop. for (;;) { // Spawn a new thread that runs the <server> entry point. // Handle UNIX/Win32 portability. #if defined (_WIN32) #if defined (_WIN32) CreateThread (0, 0, LPTHREAD_START_ROUTINE(&logging_handler), SOCKET h; reinterpret_cast <void *> (h), 0, &t_id); DWORD t_id; #else #else thr_create (0, 0, logging_handler,reinterpret_cast <void *> (h), int h; THR_DETACHED, &t_id); thread_t t_id; #endif /* _WIN32 */ #endif /* _WIN32 */ } return 0; } Dein Name – Dein Vortragstitel Pra 01 - S. 41
  • 42. Advanced Developers Conference 10.-11. Oktober 2004 Wrapper Facade: Bemerkungen Structure Solution Structure: Encapsulate functions Wrapper Client Facade and low-level data structures within method() classes: Functions are existing low-level functions Functions and data structures. function() Wrapper Facades shield clients from dependencies to the functions by class INET_Addr { public: INET_Addr (u_short port, u_long addr = 0) { providing methods that forward client // Set up the address to become a server. memset (&addr_, 0, sizeof addr_); invocations to the Functions. addr_.sin_family = PF_INET; addr_.sin_port = htons (port); A Client accesses the low-level functions addr_.sin_addr.s_addr = htonl (addr); } via Wrapper Facades. u_short get_port () const { return ntohs (addr_.sin_port); } u_long get_ip_addr () const { return ntohl (addr_.sin_addr.s_addr); } class SOCK_Acceptor { public: sockaddr *addr () const // Initialize a passive-mode acceptor socket. { return reinterpret_cast <sockaddr *> (&addr_);} SOCK_Acceptor (const INET_Addr &addr) { // Create a local endpoint of communication. size_t size () const { return sizeof (addr_); } handle_ = socket (PF_INET, SOCK_STREAM, 0); Bemerkungen // Associate address with endpoint. // ... bind (handle_, addr.addr (), addr.size ()); private: // Make endpoint listen for connections. sockaddr in_addr; listen (handle_, 5); }; }; // ... }; Wrapper Facade: Dynamics Solution Dynamics: Forward method invocations on a Wrapper Facade to the Wrapper Client Function appropriate low-level Functions: Facade method A Client invokes a method on a Wrapper function Bemerkungen Facade. The Wrapper Facade executes one or more low-level Functions in the right order, thereby performing all necessary data conversions and error handling. Dein Name – Dein Vortragstitel Pra 01 - S. 42
  • 43. Advanced Developers Conference 10.-11. Oktober 2004 Wrapper Facade: Bemerkungen Consequences Benefits Robustness: Low-level APIs are used correctly. Portability: An application depends on unified, semantically expressive interfaces, not on low-level API syntax. Modularity: Wrapper Facades are low-level, self- contained building blocks. Liabilities Indirection: Wrapper Facades add a level of indirection between application code and API code. Bemerkungen Wrapper Facade: Example The open source Bemerkungen ADAPTIVE Communication Environment (ACE) ACE provides a rich set of Wrapper Facades that abstract from more than 40 operating systems: Sockets Threads Processes Messages Synchronization www.cs.wustl.edu/~schmidt/ACE.html ... Dein Name – Dein Vortragstitel Pra 01 - S. 43
  • 44. Advanced Developers Conference 10.-11. Oktober 2004 Wrapper Facade: Known Bemerkungen Uses Class libraries and frameworks: Rouge Wave Net.h++, Rouge Wave Threads.h++, ACE, to abstract from operating system APIs. .NET and Java Virtual Machines to abstract from the underlying OS. Graphical User Interfaces Libraries: Win Forms, Smalltalk, MacApp, InterViews, Unidraw, NIHCL, ET++, MFC, Swing to abstract from low-level GUI libraries. Bemerkungen Wrap-Up Patterns helped us realizing a highly Virtual_ Evictor LFU Filesystem efficient yet flexible web server: Reactor defines the fundamental event Abstract_ LRU Guard Eviction handling architecture of the web server. Acceptor-Connector improves availability by Thread_ separating connection establishment from Bemerkungen shares Pool dispatches HTTP processing join() leader() Leader/Followers and Monitor Object provide * (de) a high-performance concurrency architecture. Event_ Reactor activates Handler Caching and an Evictor with associated maintains owns eviction Strategies speed up accessing * * * frequently used URLs. select() Handle notifies Singleton ensures the uniqueness of the server‘s virtual file system. HTTP_ HTTP_ Scoped Locking, Strategized Locking, and Acceptor Handler Double-Checked Locking ensure that the virtual file system is thread-safe without Thread_ Condition_ incurring locking overhead. Socket Thread Mutex Variable Wrapper Facade ensures portability. Dein Name – Dein Vortragstitel Pra 01 - S. 44
  • 45. Advanced Developers Conference 10.-11. Oktober 2004 Bemerkungen Further Refinements Bemerkungen Abstractions (I) When developing a web server different layers of abstractions are available. For example: Handling network protocols from the physical medium up to its handling in the web server. Bemerkungen The functionality in the web server covers different concerns such as communication, media handling, file system access, security issues, …. If all concerns are interwoven it is difficult to change, maintain, or extend the web server because no one knows what implications a This was straightforward change might have. to build, but how can I now Putting everything in components is change it? not sufficient Dein Name – Dein Vortragstitel Pra 01 - S. 45
  • 46. Advanced Developers Conference 10.-11. Oktober 2004 Bemerkungen Abstractions (II) Structure the web server . implementation into different abstraction using Layers: . Layers of abtraction such as the . TCP/IP layer are separated into different layers of implementation. Security Layer This way, you might even change the protocol stack implementation without impact on other system HTTP Layer parts. The Wrapper Facades introuced in TCP/IP Layer previous slides would also build one or more layers of abstraction Bemerkungen Abstractions (II) Virtual_ Evictor LFU Shield the web server Filesystem implementation from low-level Abstract_ OS-APIs via a set of Wrapper LRU Guard Eviction Facades: Thread_ A set of wrapper facades (Socket, Bemerkungen shares Pool dispatches join() Thread, Thread_Mutex, leader() * Condition_Variable) provide a (de) Event_ Reactor activates portable, robust, and modular Handler maintains owns access to low-level OS APIs. * * * select() Handle Application classes (our web server notifies classes) use the wrapper facades in HTTP_ HTTP_ their implementation. Acceptor Handler Thread_ Condition_ Socket Thread Mutex Variable Dein Name – Dein Vortragstitel Pra 01 - S. 46
  • 47. Advanced Developers Conference 10.-11. Oktober 2004 Bemerkungen Layers: Problem Layers (POSA1) Problem: Some applications consist of services User Interface that reside at different abstraction levels, with the higher-level services building upon the lower- Biz Svc Biz Svc level services. However: Despite their cooperation relationships and Biz Obj Biz Obj Biz Obj dependencies, services of different abstraction levels should be decoupled as much as possible. Resource Mgmt DB-Access Yet not all services can exist in isolation. Instead they strongly complement one another. Services at a particular level of abstraction may OS Database also evolve but changes should not ripple through the entire application and all its Bemerkungen abstraction levels. Layers: Structure Solution Structure: Decouple User Interface the abstraction levels of an UI_svc_1 UI_svc_2 UI_svc_3 Layer application by splitting it into several loosely connected layers—one layer for each Bemerkungen Business abstraction level. BP_svc_1 BP_svc_2 BP_svc_3 Process Layer Services realize the application’s functionality. Layers comprise all services Business BO_1 BO_2 BO_3 Object Layer of a particular abstraction level. Database DA_svc_1 DA_svc_2 DA_svc_3 Access Layer Dein Name – Dein Vortragstitel Pra 01 - S. 47
  • 48. Advanced Developers Conference 10.-11. Oktober 2004 Bemerkungen Layers: Dynamics Solution Dynamics: Let services of a particular Layer1 Layer1 Layer2 Layer3 Service2 Service2 Service Service layer call only services of lower-level layers. service service service Bemerkungen Layers: Consequences Benefits Loose coupling: Layers are decoupled from one another as much as possible. Exchangeability: Layer implementations can be exchanged transparently for other layers. Bemerkungen Reusability: Layers and their services are units of reuse. Liabilities Lower efficiency: The more layers there are, and the more decoupled these layers are, the less efficient is the system! Layer granularity: Finding the right layer granularity is often non-trivial. Dein Name – Dein Vortragstitel Pra 01 - S. 48
  • 49. Advanced Developers Conference 10.-11. Oktober 2004 Bemerkungen Layers: Example Layer 7: Provides OSI 7-Layer Model: The International miscellaneous protocols Application Standardization Organization (ISO) for common activities defined a 7-layer architecture for Layer 6: Structures Presentation networking protocols. information and attaches semantics Layer 5: Provides dialog Session control and synchronization Layer 4: Breaks messages Transport into packets and guarantees delivery Layer 3: Selects a route Network from sender to receiver Layer 2: Detects and Data Link corrects errors in bit sequences Layer 1: Transmits bits Physical Bemerkungen Layers: Known Uses OSI 7-Layer Model for networking protocols. Presentation Tier N-Tier applications follow a layered architecture where layers DOC-Middleware are distributed across multiple Bemerkungen network nodes. Business Tier Windows NT: NT defines five layers: system services, resource Bizz Bizz Bizz management, kernel, HAL Comp1 Comp2 Comp3 (Hardware Abstraction Layer), Hardware DOC-Middleware Database Tier Dein Name – Dein Vortragstitel Pra 01 - S. 49
  • 50. Advanced Developers Conference 10.-11. Oktober 2004 Out-of-the-band Bemerkungen Functionality (I) When developing a web server we cannot anticipate what kind of New functionality? functionality might required: People might want to add additional media formats. Other network protocols such as SOAP might be required. Plug-Ins should be supported. The processing flow should be be modifyable. One way would be to let the development team of the web-server add those things exclusively. In this scenario, we end up with a whole ball of mud since the possible extensions would flood the system. Otherwise, we could stay with minimal extensions, but then people would very likely switch to other products. Bemerkungen Out-of-the-Band Functionality (II) Open the web server implementation according to the Open-Close principle using waiting Interceptors: Define hooks in the processing state new request Bemerkungen 3rd parties might machine where new request add code. Registered interceptors get automatically by the web server Interceptor when specific events occur. Users might add their interceptors to midify and extend the processing logic for adding new media formats, protocols, plug-ins, security processing features, ... Interceptors might be combined by Piples & Filters or Chain of Responsibility. Dein Name – Dein Vortragstitel Pra 01 - S. 50
  • 51. Advanced Developers Conference 10.-11. Oktober 2004 Bemerkungen Interceptor: Problem Interceptor (POSA2) Problem: Some client applications may require more functionality from an application than it originally includes or is intended to include. However: And we It is impossible to anticipate specific Transactions! We Load requirements of applications a priori, Balancing! specifically for application frameworks and product families. The implementation of a customer- specific system version should not pollute the code of other system versions that not need its specific Bemerkungen We need But we services. Security! don’t! Interceptor: Structure Interceptor Solution Structure: Open an implements callback Callback implementation so that external services interface interface can react on the occurrence of internal A Framework events: A Framework provides application functionality Bemerkungen Point Interception and specifies internal events on which occurrence add-on functionality can be Client triggered. Context uses An Interceptor declares a general interface for set_value() 1..* handling these events. get_value() Conc-Interceptor Concrete Interceptors implement the creates callback_1() interceptor interface in a client-specific way. Framework callback_2() Dispatchers allow to register interceptors with register/ unregister 1..* the framework and notify them when events interceptors occur. Dispatcher Interceptor Context Objects provide interceptors with 1..* (un)register() callback_1() information about an event occurred. dispatch() callback_2() Dein Name – Dein Vortragstitel Pra 01 - S. 51
  • 52. Advanced Developers Conference 10.-11. Oktober 2004 Bemerkungen Interceptor: Dynamics Solution Dynamics: Call back all Frame- Client registered interceptors whenever a work framework-internal event of interest Concrete Dispatcher Interceptor occurs: create Conc. create Interceptor A Client creates a Concrete Interceptor register event and registers it with a Dispatcher. create Context Object In the Framework, an event occurs that Context Object is passed via the Dispatcher to the Context callback Object Concrete Interceptor, along with a get_value Context Object containing information set_value set_value about the event. The Concrete Interceptor uses the information it receives from the Context Object to execute its functionality, if necessary by modifying the Framework‘s state through the Context Object‘s Bemerkungen accessor methods. Interceptor: Consequences Benefits Extensibility and Flexibility: A design can be prepared for unexpected extensions without destabilizing it. Separation of concerns: Core functionality is separated from out-of-band functionality. Bemerkungen Control: Ideal support for debugging, monitoring, tracing, logging etc. Reusability: Lean functionality is better reusable than fat functionality. Liabilities Complexity: High realization effort. Stability: If interceptors break, the framework can break too; malicious interceptors. Limited power in most languages: OO is not the most ideal programming paradigm. Dein Name – Dein Vortragstitel Pra 01 - S. 52
  • 53. Advanced Developers Conference 10.-11. Oktober 2004 Bemerkungen Interceptor: Example .NET Remoting specifies Interceptors that are designed to intercept the flow of a request/reply sequence through the runtime at specific points so that services can query the request information and manipulate the service contexts which are propagated between clients and servers. Bemerkungen Interceptor: Known Uses .NET Remoting, CORB specify portable interceptors to intercept the ORB communication between clients and servers. Reflective Systems: Interceptors Bemerkungen intercept important events occurring in the base-level, Joinpoint inspect state information from Declaration aspect PublicTracing { Log log = new Log(); the base-level, and influence its pointcut publicMethodCalls (): subsequent behavior. calls(public * com.expressions.*.*(..)); before() : publicMethodCalls() { Aspect weavers weave aspects Trace.traceEntry (thisJoinPointStaticPart.getSignature().toString); into code that (conceptually) } intercept the system’s control Advice refers after() : publicMethodCalls() { Trace.traceExit flow at certain join-points to to joinpoint (thisJoinPointStaticPart.getSignature().toString); } execute additional behavior. } Dein Name – Dein Vortragstitel Pra 01 - S. 53
  • 54. Advanced Developers Conference 10.-11. Oktober 2004 Bemerkungen Runtime Configuration (I) A web server implements different strategies wjich might be exchangeable such as the concurrency or dispatching models used: It should be possible to exchange strategies at runtime. It would also be avantageous to add functionality dynamically. Seldomly used functionality could be dynamically added or removed. However, these changes should have no impact on the runtime execution. Bemerkungen Runtime Configuration (II) Add new functionality bs dynamically reconfiguring the web server using the Component Configurator: Exchangeable components provide a specific interface and can be Bemerkungen dynamically added/removed. The lifecycle of these components is controlled by a component configurator within the web server. Now, we can easily change strategies if we implement them by components. Dein Name – Dein Vortragstitel Pra 01 - S. 54
  • 55. Advanced Developers Conference 10.-11. Oktober 2004 Component Configurator: Bemerkungen Problem Component Configurator (POSA2) Problem: It may be necessary to re-configure a system with a different set of service components than done in its original configuration. However: The re-configuration must happen at run-time. Different end user needs and traffic workloads require configurable Services must be initialized, web server architectures regarding many aspects: suspended, resumed, and • Concurrency models: thread per request, thread pool, ... • Event demultiplexing models: synchronous or asynchronous terminated dynamically. Bemerkungen LFU, ... • File caching models: LRU, • Content delivery protocols: HTTP/1.0, HTTP/1.1, HTTP-NG Component Configurator: Structure Solution Structure: Separate the configuration of components from the Component Component Configurator Repository point in time they are used. install() insert() A Component defines a common uninstall() remove() interface for all configurable installs maintains Bemerkungen components. * Concrete Components implement Component specific configurable components. init() A Component Repository maintains the fini() suspend() currently configured and loaded resume() components. info() service() A Component Configurator can dynamically load components to, and unload from, the component repository, Concrete Concrete and execute them. Component1 Component2 Dein Name – Dein Vortragstitel Pra 01 - S. 55
  • 56. Advanced Developers Conference 10.-11. Oktober 2004 Component Configurator: Bemerkungen Dynamics Solution Dynamics: Let the Component Component Clients/ Component Component Configurator control the Configurator A Services Repository (re-) configuration of an application: install The Component Configurator loads all init init concrete components of a particular configuration, initializes them and inserts them into the Component service Repository. The Concrete Components serve client requests. uninstall fini When re-configuring an application, suspend the Component Configurator terminates all Concrete Components and removes them from the Component Repository. Bemerkungen Component Configurator: Consequences Benefits • Centralized administration: All components are managed centrally. • Increased configuration dynamism: Components can be (re-) configured without shutting a system down and at almost any point in time during system execution. Bemerkungen • Availability: During (re-) configuration an application is still partially available. Liabilities • Lack of determinism: Configuration is done at run-time. • Structural and behavioral overhead: A Component Configurator arrangement results in a more complex system and component architecture and in more complex behavior. Dein Name – Dein Vortragstitel Pra 01 - S. 56
  • 57. Advanced Developers Conference 10.-11. Oktober 2004 Component Configurator: Bemerkungen Example JAWS Web Server Framework: Virtual_ Evictor LFU Filesystem JAWS uses the Component Configurator pattern to dynamically Abstract_ LRU Guard Eviction optimize, control, and reconfigure the behavior of its Web server Thread_ Pool shares dispatches strategies at installation-time or join() leader() during run-time. For example, * (de) Event_ JAWS applies the Component Reactor activates Handler maintains owns Configurator pattern to configure its * * * various Cached Virtual Filesystem select() Handle notifies strategies, such as least-recently used (LRU) or least-frequently used HTTP_ HTTP_ Acceptor Handler (LFU). Thread_ Condition_ Socket Thread Mutex Variable Bemerkungen Component Configurator: Known Uses ADAPTIVE Communication Environment (ACE): ACE provides a reusable Component Configurator framework – the service Bemerkungen configurator – for self- contained network services. Dein Name – Dein Vortragstitel Pra 01 - S. 57
  • 58. Advanced Developers Conference 10.-11. Oktober 2004 Bemerkungen Annotated References Bemerkungen The Gang of Four Book The Gang Of Four Book is the first, and still the most popular pattern book. It contains 23 general purpose design patterns and idioms for Object creation: Abstract Factory, Builder Factory Method, Prototype, and Singleton Bemerkungen Structural Decomposition: Composite and Interpreter Organization of Work: Command, Mediator, and Chain of Responsibility Service Access: Proxy, Facade, and Iterator Extensibility: Decorator and Visitor Variation: Bridge, Strategy, State, and Template Method Adaptation: Adapter Resource Management: Memento and Flyweight Communication: Observer Dein Name – Dein Vortragstitel Pra 01 - S. 58
  • 59. Advanced Developers Conference 10.-11. Oktober 2004 Bemerkungen A System Of Patterns A System Of Patterns is the first volume of the POSA series and the second most popular pattern book. It contains 17 general purpose architectural patterns, design patterns and idioms for: Structural Decomposition: Layers, Blackboard Pipes and Filters, and Whole Part Distributed Systems: Broker, Forwarder- Receiver and Client-Dispatcher-Server Interactive Systems: Model-View-Controller and Presentation-Abstraction-Control Adaptive Systems: Microkernel, Reflection Organization of Work: Master Slave Service Access: Proxy Resource Management: Counted Pointer, Command Processor, View Handler Bemerkungen Communication: Publisher-Subscriber Patterns For Concurrent and Networked Objects Patterns For Concurrent And Networked Objects is the second volume of the POSA series. It contains 17 architectural patterns, design patterns and idioms for concurrent, and networked systems: Service Access and Configuration: BemerkungenConfigurator, Wrapper Facade, Component Interceptor and Extension Interface Event Handling: Reactor, Proactor, Asynchronous Completion Token and Acceptor-Connector Synchronization: Scoped Locking, Double- Checked Locking, Strategized Locking, and Thread-Safe Interface Concurrency: Active Object, Monitor Object, Leader/Followers, Thread-Specific Storage Half-Sync/Half-Async Dein Name – Dein Vortragstitel Pra 01 - S. 59
  • 60. Advanced Developers Conference 10.-11. Oktober 2004 Patterns for Resource Bemerkungen Management Patterns For Resource Management covers patterns helpful for resource acquisition and management. It is structured as a pattern language and includes patterns such as • Evictor • Lease • Lazy Evaluation • Caching • Lookup • ... Bemerkungen Server Component Patterns Server Component Patterns includes a pattern language that illustrates core design concepts for containers as well as fundamental design criteria for components. Bemerkungen outlined how it is For each pattern is implemented in EJB, CCM, and COM+ A separate part in the book describes the EJB implementation in full depth. Dein Name – Dein Vortragstitel Pra 01 - S. 60
  • 61. Advanced Developers Conference 10.-11. Oktober 2004 Enterprise Architecture Bemerkungen Patterns Patterns of Enterprise Architecture includes a pattern language with about 50 patterns that illustrates how to design 3-tier enterprise business information applications. A core strength of the book is its fine collection of patterns to map from an object-oriented application to a relational database. Bemerkungen Bemerkungen Dein Name – Dein Vortragstitel Pra 01 - S. 61