Queueing
Upcoming SlideShare
Loading in...5
×
 

Queueing

on

  • 1,978 views

 

Statistics

Views

Total Views
1,978
Views on SlideShare
1,974
Embed Views
4

Actions

Likes
1
Downloads
58
Comments
0

2 Embeds 4

http://61.17.42.159:5050 3
http://203.144.133.44 1

Accessibility

Categories

Upload Details

Uploaded via as Microsoft PowerPoint

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

Queueing Queueing Presentation Transcript

  • Performance Engineering Prof. Jerry Breecher QUEUEING
  • WHAT WE ARE DOING HERE We are about to embark on: Queueing Lingo - all the definitions you’ll ever need! Queueing - especially the Single Queue in DETAIL Analytical Models - how to solve a number of complex problems using the equations we already know and love. Works especially well for open models. Operational Laws - drawing conclusions about limits - what to do when we can’t solve the math exactly. Mean Value Analysis - Using the equations in an iterative fashion to solve closed models.
  • Queueing Models
    • This section is about being able to describe the behavior of queues. Queues are certainly a prevalent object in computer systems, and our goal here is to write the equations that describe them. The language we’ll use here is mathematical, but nothing really more complicated than algebra.
  • Queueing Lingo
    • Goals:
      • To understand the random/statistical nature of computer data. We will emphasize the non-deterministic.
      • To understand distributions for simulation purposes.
      • To impress your friends.
    • For an observation, two things matter:
      • The value measured.
      • When measured.
    • The occurrence of an event can give us the "when".
  • Queueing Lingo
    • A STOCHASTIC PROCESS is a mechanism that produces a collection of measurements which all occur, randomly, in the same range of values. It applies to the VALUE measured for an observation. The dictionary says, "random, statistical".
    • Stochastic processes are well behaved phenomena which don't do things which are unpredictable or unplanned for.
    • Examples:
    • Throwing 2 dice always gives numbers in the range 2 - 12.
    • Actions of people are unpredictable (unless the range of values is made very large.) Someone can always respond in a way you haven't predicted.
  • Queueing Lingo
    • THE POISSON PROCESS applies to WHEN an observation is made. It looks random; the arrival points are uniformly distributed across a time interval. Poisson processes can be defined by:
    • Event counting The distribution of the number of events occurring in
    • a particular time is a Poisson distribution.
    • Time between events The distribution of times between event occurrences is
    • exponential.
    • Example: Show how a random "look" leads to an exponential distribution. See the next page for a picture of these distributions.
  • Queueing Lingo
    • This is a simple exponential curve. What properties can you identify from it?
    F(t) = exp(-t)
  • Queueing Lingo
    • Example of the Poisson Probability Density Function.
    F(k) = ( 5 / k! ) exp( -5 )
  • Lab - You Get To See It Happen Before Your Very Eyes!
    • This is a group lab designed to demonstrate that the number of random events in a particular interval is a Poisson distribution, and that the distribution of spaces between events is exponential.
    • Open your textbook 20 times. Each time, write down the number of the left-hand page ( mod 100 ) beside the appropriate sample in column I.
    • Count how many of the segments in Column I have 0 samples in them. Put this number in the first row of Column II. Repeat for 1, 2, 3, ... samples.
    • For each of the samples in Column I, determine the distance or separation between that sample and the next higher sample. If the the distance is 0 ( the two samples are the same ) put a tick in the first row of column 3, if the separation is 3, tick the interval = 3 row, etc.
  • Lab - You Get To See It Happen Before Your Very Eyes! Column I Numbers falling in each segment. Column II Number of segments containing N samples Column III Number of instances of intervals between samples. Segment How Many Samples 0 – 9   10 – 19   20 - 29   30 - 39   40 - 49   50 - 59   60 - 69   70 - 79   80 - 89   90 - 99   N Number of Segments 0   1   2   3   4   5   6   7   8   9   Interval Number of Instances 0   2   4   6   8   10   12   14   16   18  
  • Lab - You Get To See It Happen Before Your Very Eyes! // Generate Poisson. // This program generates random numbers, puts them into buckets, // and calculates the distance between numbers. // // Inputs: // N - number of samples to generate #include <time.h> #define MAX_DATA_VALUE 100 #define MAX_BUCKETS 100 #define MAX_DATA 1000 // Compare routine used in sorting int compare (const void * a, const void * b) { return ( *(int*)a - *(int*)b ); } int main( int argc, char *argv[]) { int Data[MAX_DATA]; int Bucket[MAX_BUCKETS]; int NumberOfSamples; int NumberOfBuckets = 10; int LargestBucketFill = 0; int Index, Temp, i; if ( argc <2 ) { printf( &quot;Usage: GeneratePoisson <number_of_samples> &quot;); exit(0); } NumberOfSamples = atoi( argv[1] ); srand((unsigned)time(NULL)); for ( Index = 0; Index < NumberOfBuckets; Index++ ) Bucket[Index] = 0; for ( Index = 0; Index < NumberOfSamples; Index++ ) { Data[Index] = rand() % MAX_DATA_VALUE; Temp = (NumberOfBuckets*Data[Index])/MAX_DATA_VALUE; Bucket[Temp]++; if ( Bucket[Temp] > LargestBucketFill ) LargestBucketFill = Bucket[Temp]; } printf(&quot; Raw Random Numbers &quot;); for ( Index = 0; Index < NumberOfSamples; Index++ ) printf( &quot;%d &quot;, Data[Index] ); printf(&quot; &quot;); qsort (Data, NumberOfSamples, sizeof(int), compare); printf( &quot; Sorted Random Numbers &quot;); for ( Index = 0; Index < NumberOfSamples; Index++ ) printf( &quot;%d &quot;, Data[Index] ); printf(&quot; &quot;); printf( &quot; Range Number &quot;); printf( &quot; in Range &quot;); for ( Index = 0; Index < NumberOfBuckets; Index++ ) printf( &quot;%2d - %2d, %d &quot;, (MAX_DATA_VALUE/NumberOfBuckets * Index), (MAX_DATA_VALUE/NumberOfBuckets * (Index+1)) - 1, Bucket[Index] ); // Calculate distribution of items in each bucket printf(&quot; &quot;); for ( Index = 0; Index < LargestBucketFill; Index++ ) { Temp = 0; for ( i = 0; i <NumberOfBuckets; i++ ) if ( Bucket[i] == Index ) Temp++; printf( &quot;Number of buckets with %d items, %d &quot;, Index, Temp ); } // Determine the distance between the random items. for ( Index = 0; Index < MAX_BUCKETS; Index++ ) Bucket[Index] = 0; LargestBucketFill = 0; for ( Index = 0; Index < NumberOfSamples - 1; Index++ ) { Temp = Data[Index+1] - Data[Index]; Bucket[Temp]++; if ( Temp > LargestBucketFill ) LargestBucketFill = Temp; } printf( &quot; Distance Number &quot;); printf( &quot; Between with this &quot;); printf( &quot; Samples distance &quot;); for ( Index = 0; Index <= LargestBucketFill; Index++ ) printf( &quot;%2d, %d &quot;, Index, Bucket[Index] ); } Code is here so it doesn’t get lost!!
  • Lab - You Get To See It Happen Before Your Very Eyes! The Source Code The results of running this code can be seen here  This is the result of running 4 experiments of 20 samples each. The results aren’t pretty!! Perhaps by running the tests many times, everything would look nice and smooth. You can see the results you would expect, but there’s lots of jitter. Experimental Results
  • Queueing Lingo
    • Examples:
    • Suppose that a piece of software has an expected lifetime of 5 years, and that the average bug rate for this type of product is one bug/year.
    • What is the bug expectation rate per year at the start of the five years, assuming this code is &quot;average&quot;?
    • After two years, four bugs have been found. The code is still considered &quot;average&quot;. How many bugs/year can be expected for the remaining three years?
  • Queueing Lingo
    • MEMORYLESS means that the probability of an event doesn't depend on its past. The above case highlights an example where the past does matter.
    • Examples : Which depend on the past, and which don't?
      • Throwing dice?
      • A disk seek distance?
      • The address of an instruction execution?
      • The measurement of the length of a table?
    Prepare to Have Your Mind Bent In An Unexpected Way!! Example: Consider a bus stop where the time between bus arrivals is exponentially distributed with a rate L. Thus the bus arrivals form a Poisson process. If you walk up to the bus stop, how long do you have to wait until the next bus arrives?
  • Queueing Lingo
    • Example:
    • Consider a bus stop where the time between bus arrivals is exponentially distributed with a rate L. Thus the bus arrivals form a Poisson process. If you walk up to the bus stop, how long do you have to wait until the next bus arrives?
    • 1. Possible solution: Since buses arrive at a rate L, the average time between arrivals is 1/L. But since we walk up at random, we would wait for only half an interval on the average. So we would wait 1/(2L) for the next bus.
    • 2. Possible solution: Since the time between buses is exponentially distributed, it is memoryless. So the residual lifetime for any time that I arrive should be distributed exponentially the same way as the original distribution. Since the average time between buses is 1/L, the average time ( residual ) to wait should also be 1/L.
  • Queueing Lingo
    • // GenerateRandomBusArrivals
    • #include <time.h>
    • #define MAX_DATA 1000
    • // Hours in a week
    • #define MAX_DATA_VALUE 168
    • // Compare routine used in sorting
    • int compare (const void * a, const void * b) {
    • return ( *(int*)a - *(int*)b );
    • }
    • int main( int argc, char *argv[]) {
    • int Data[MAX_DATA];
    • int NumberOfSamples = 14;
    • int Index;
    • printf( &quot;This program generates 1 week's worth of bus arrivals. &quot;);
    • printf( &quot;We assume that the average bus arrival rate is every 12 hours, &quot;);
    • printf( &quot;The arrival rate is 1/12 per hour. We'll take 1 week's worth &quot;);
    • printf( &quot;of arrivals - we will assume that there are 14 arrivals in a week &quot;);
    • printf( &quot;- that's how we set the average arrival rate. &quot;);
    • printf( &quot;Oh - and we assume the buses arrive on the hour (it's simpler that way). &quot;);
    • srand((unsigned)time(NULL));
    • for ( Index = 0; Index < NumberOfSamples; Index++ ) {
    • Data[Index] = rand() % MAX_DATA_VALUE;
    • }
    • qsort (Data, NumberOfSamples, sizeof(int), compare);
    • printf( &quot; Sorted Random Numbers &quot;);
    • for ( Index = 0; Index < NumberOfSamples; Index++ )
    • printf( &quot;%d &quot;, Data[Index] );
    • printf(&quot; &quot;);
    • }
    This program generates 1 week's worth of bus arrivals. We assume that the average bus arrival rate is every 12 hours, The arrival rate is 1/12 per hour. We'll take 1 week's worth of arrivals - we will assume that there are 14 arrivals in a week - that's how we set the average arrival rate. Oh - and we assume the buses arrive on the hour (it's simpler that way). Sorted Random Numbers 2 5 73 81 100 100 102 109 134 136 145 148 152 154 So given these actual bus arrivals, how long on average must you wait for a bus? The Source Code
  • Queueing Lingo
    • Types of Stochastic Processes
    Arrival rates are often Poisson. Service times are often exponential. ( In other words, they're both random. )
  • PROPERTIES OF QUEUES The device doing the actual service of the customers. Customer Departures The queue – A place where customers are stored before being serviced. Customer Arrivals
  • PROPERTIES OF QUEUES
    • How do we describe a queue? These are the important aspects:
    • Arrival process:
    • Service Time Distribution:
    • Number of Servers:
    • System Capacity :
    • Population Size:
    • Service Discipline:
    • The shorthand for queue description is thus A / S / m / B / K / SD.
    • The inter-arrival and service times are typically of the following types:
    • M Exponential – Memoryless, the distribution we’ve just been talking about.
    • D Deterministic – the times are constant and there is no variance.
    • G General –distribution is not specified and the results are valid for all distributions
  • THE SINGLE QUEUE
    • If we have a single queue obeying certain properties, we can get all kinds of nice metrics. But, it must have those required properties!!
    • REQUIRED PROPERTIES:
    • Arrivals are random with a rate of X per time. ( Poisson – when we say this, we mean the inter-arrival time is exponentially distributed. ) [ Note that in steady state, throughput = arrival rate.] Many texts use  for this.
    • Service times are random with a value of D. (Exponential ) [ Note this is the Demand we've seen before.] Many texts use  for this. The rate of service is  = 1/D.
    • There's the possibility of an infinite number of customers.
    • There's a single server. [ So the derivation we’re about to do doesn't work for a multiprocessor CPU.]
  • THE SINGLE QUEUE
    • These are general requirements and hold for many practical applications. Other analysis can be done for cases outside these requirements, but we don't do it here.
    • We will apply a method called local balance to a system like that pictured on the next page:
    • The queue is of type M / M / 1.
  • THE SINGLE QUEUE
    • For simplification, in this particular case, the utilization U is related to throughput and demand by
    • U = X D (Remember N = XS)
    • Note: U 
        • p i = U , p 0 = ( 1 – U )
    X =  X =   = 1/D  = 1/D State with 0 in Queue State with 1 in Queue State with 2 in Queue
  • THE SINGLE QUEUE
    • By Definition: A queue is defined to contain customers that are both waiting and being serviced.
    • In an equilibrium state, from the picture below, the following equations can be formed:
    •  p i =  p i-1
    • p    p i 
    • p      p 0  U    p 0
    • The probability of having i customers in the queue is
    • p i = ( 1 – U ) U i
    • [ Note that p 0 = ( 1 - U ) so p i > 0 = U . But this is just the utilization we defined before.]
    X =  X =   = 1/D  = 1/D State with 0 in Queue State with 1 in Queue State with 2 in Queue
  • THE SINGLE QUEUE
    • The average number of customers in the queue (waiting and being serviced) is
    • From Little's Law ( N = X T ) in steady state, we can derive the average time spent at the queueing center ( both in the queue and being serviced ). Note what happens to this response time as the utilization increases!
  • THE SINGLE QUEUE
    • Example:
      • Pat is designing a communications server that receives requests from &quot;higher level&quot; routines. The requests are collected by a Request Handler that does nothing but put them into buffers. These requests are removed from the buffers by the Request Processor on a first come first serve basis.
      • requests -> Request Handler -> Buffers[n] -> Request Processor ->
      • The requests arrive randomly at a rate of 5/second. The Request Processor can service 10 items per second from the buffer.
      • Since allocating buffers is an expensive business, Pat wants to preallocate an adequate number of buffers so none need be allocated 99% of the time. Clearly there's a tradeoff here between memory usage and time-to-allocate.
      • How many buffers should be preallocated?
  • THE SINGLE QUEUE X =   = 1/D State with 0 in Queue State with 1 in Queue This is the setup for the case of M / M / 2. The probability of transition from a lower population to a higher one is the same as before ( arrivals are the same.) But the probability of one of the TWO servers finishing is twice as great when both of them are filled. X =   State with 2 in Queue X =   State with 3 in Queue
  • THE SINGLE QUEUE The following equations hold for the single queue.
  • ANALYTICAL MODEL
    • Goals:
      • You should be able to create, use, and understand a simple analytical model.
      • You should have a general idea of when such models are applicable and should understand some of the buzzwords.
    • PERSPECTIVE:
    • An Analytical Model uses mathematical operations to capture the relationships between observable quantities. The computations don't necessarily mimic real actions as they do for simulations.
    • Examples:
      • The equations we've been using such as Little's Law.
      • Local Balance Equations - these enumerate the states the system can be in and then determine the transitions between states. (This is what we just did with single queues.)
      • Mean Value Analysis - an iterative approach using the equations we've already learned.
  • ANALYTICAL MODEL
    • Analytical models can be applied to:
      • Single Queues
      • Queueing Networks
    • Queueing Networks are networks of queues; two or more queues tied together. They can be:
      • Open: - Typical of transaction processing. Jobs enter and leave the system being studied.
      • Closed : - typical of batch or terminal systems. Jobs always remain somewhere within the system.
      • Single Class - the customers are indistinguishable from each other; they have similar service demands and routing characteristics.
      • Multiple Class - several categories of customers can be identified. A batch class, for example, might be heavily CPU bound while a terminal class is I/O bound. To use a multiple class model requires determining the characteristics for EACH of the classes involved.
  • ANALYTICAL MODEL SINGLE CLASS OPEN QUEUEING NETWORK MODEL SOLUTIONS: This is EASY!! It's simply an extension of the equations we used for the single queue. Utilization: U k = X D k {D k is the service time) Throughput: X k = X V k (Throughput = arrivals) Max. Throughput: X max = Residence Time: R k = (delay centers) (queueing centers)
  • ANALYTICAL MODEL
    • SINGLE CLASS OPEN QUEUEING NETWORK MODEL SOLUTIONS:
    • Queue Length: Q k = (delay centers)
    • (queueing centers)
    • System Response T = R =
    • Time:
    • Average Number N = Q =
    • In System:
    • Remember,
        • N = Number of requests in the &quot;system&quot;
        • X = Throughput
        • R = Residence time per request.
        • S = Service Time
    • Little's law is 90% of all you'll ever need.
    We'll be using the FIGURE on the next page and applying these Laws to the system shown there, at a number of different levels.
  • ANALYTICAL MODEL Terminals 4 3 2 1 CPU DISK A DISK B DISK C
  • ANALYTICAL MODEL
    • Box 1 in the Figure: A single resource, not including the queue.
    • Here the population, N, is either 1 or 0 ( in use or not ).
        • Utilization is equal to the average number of requests present, or the average N.
        • Throughput X is the number of requests serviced per time.
        • Residence time R is, for this case, the service time.
    • [NOTE: Little's Law reduces to Utilization Law.]
    1 DISK C
    • Example:
      • Suppose a disk, serves 100 requests/second, with the average request needing 0.008 seconds of disk service.
      • What is the utilization of the disk?
    N = X * R U = X * S R = S/(1 – U )
  • ANALYTICAL MODEL
    • Box 2 in the Figure: A single resource, including the queue.
    • Now the population includes both those requests in the queue and in service.
        • Throughput remains the rate that the resource satisfies requests.
        • Residence time is the sum of queueing and service times.
    • Example:
        • You will need to use the result from the previous slide.
        • What is the time a request spends at the disk subsystem ( spindle + queue)?
        • What is the average number of requests in “Box 2”?
        • For one of these requests, what is the average queueing time, and what is the average service time?
    2 DISK C N = X * R U = X * S R = S/(1 – U )
  • ANALYTICAL MODEL
    • Box 3 in the Figure: Central Subsystem, not including terminals.
    • The population now is the number of users with activity in the system - those not &quot;thinking&quot;.
        • Throughput is the rate that requests flow between system and users.
        • Residence time is now what we call response time.
    • Example:
      • The average system throughput (entering & leaving Box3) is 0.5/sec. There are an average of 7.5 &quot;ready&quot; (waiting) users in the Box.
      • What is the average response time?
    3 2 CPU DISK A DISK B DISK C
  • ANALYTICAL MODEL
    • Box 4 in the Figure: Entire system, including terminals.
    • The population is the total number of users, both waiting and thinking.
        • Throughput is the rate that requests flow between system and users. (same as box 3).
        • Residence time is the sum of response time and think time.
    • Example:
        • There are 10 users with average think time of 5 seconds, and the system has average response time of 15 seconds.
        • What is the throughput?
    Terminals 4 3
  • ANALYTICAL MODEL
    • For system wide applications of Little's Law, since time represents both thinking and waiting for a response,
          • N SS = X R for Subsystem or lower.
          • N ES = X ( R + Z ) for the Entire system.
    • Example:
      • A System has 64 interactive users, with an average think time of 30 seconds. Two interactions complete on average each second.
      • What is the average response time for this system?
      • How many interactions are in the system at any time?
      • What sized System ( how many CPU's ) would be needed for this application?
    RESPONSE TIME LAW: R = N/X - Z
  • ANALYTICAL MODEL
    • Example Problems You Should Now Be Able To Do:
      • The average delay experienced by a packet when traversing a computer network is 100 msec. The average number of packets that cross the network is 128 packets/sec.
      • What is the average number of packets in transit in the network?
    • Example Problems You Should Now Be Able To Do:
      • Measurements taken during one hour from a Web server indicate that the utilization of the CPU and the two disks are: U CPU = 0.25, U disk1 = 0.35, and U disk2 = 0.30. The Web server log shows that 21,600 requests were processed during the measurement interval.
      • What are the service demands (the time used by each request) at the CPU and both disks?
      • What is the maximum throughput,
      • and what was the response time of the Web server during the measurement interval?
    • Example Problems You Should Now Be Able To Do:
      • A computer system is measured for 30 minutes. During this time, 5,400 transactions are completed and 18,900 I/O operations are executed on a certain disk that is 40% utilized.
      • What is the average number of I/O operations per transaction on this disk?
      • What is the average service time per transaction on this disk?
  • ANALYTICAL MODEL
    • Example Problems You Should Now Be Able To Do:
      • A file server is monitored for 60 minutes, during which time 7,200 requests are completed. The disk utilization is measured to be 30%. The average service time at this disk is 30 msec per file operation request.
      • What is the average number of accesses to this disk per file request?
    • Example Problems You Should Now Be Able To Do:
            • A computer system has one CPU and two disks: disk 1 and disk 2. The system is monitored for one hour and the utilization of the CPU and of disk 1 are measured to be 32% and 60%, respectively. Each transaction makes 5 I/O requests to disk 1 and 8 to disk 2. The average service time at disk 1 is 30 msec and at disk 2 is 25 msec.
      • Find the system throughput.
      • Find the utilization of disk 2.
      • Find the average service demands at the CPU, disk 1, and disk 2.
    • Example Problems You Should Now Be Able To Do:
            • An interactive system has 50 terminals and the user's think time is equal to 5 seconds. The utilization of one of the system's disk was measured to be 60%. The average service time at the disk is equal to 30 msec. Each user interaction requires, on average, 4 I/Os on this disk.
            • What is the average response time of the interactive system?
  • ANALYTICAL MODEL
    • FORCED FLOW LAW:
    • The Forced Flow Law states that the flow in all parts of a system must be consistent.
    • Suppose we count both system completions and also completions at each resource. The visit count ( visit ratio ) is defined as:
        • Resource Completion: C k
        • System Completion: C
        • Visit Count: V k = C k / C
    THE FORCED FLOW LAW: X k = V k X
  • ANALYTICAL MODEL
    • Example:
    • Suppose a system has:
        • 30 terminals ( N = 30 ).
        • 18 seconds average think time ( Z = 18 ).
        • 20 visits to a specific disk/interaction (V disk = 20 ).
        • 30% utilization of that disk (U disk = 0.30 ).
        • 25 millisecs is the average service required per visit to the disk (S disk = 0.025 sec.).
    • We want to know :
        • Disk throughput =
        • System throughput =
        • Response time =
    THE FORCED FLOW LAW: X k = V k X RESPONSE TIME LAW: R = N/X - Z
  • ANALYTICAL MODEL
    • Example:
      • Consider the problem of a spy from Burger King trying to figure out how many people are at a McDonald's. The spy can't sit inside and watch all day, so must somehow calculate the number from information obtained from outside observations. Thirty customers/hour arrive on the average ( over a long period of time ) and the average customer exits after 12 minutes.
      • Assuming that all this time is spent standing in line, what is the mean queue length in the restaurant?
      • If the Standard Deviation of the 30 customers is 3, what is the uncertainty of the queue length?
      • What happens if both the arrival rate and the service time have uncertainties?
    THE FORCED FLOW LAW: X k = V k X RESPONSE TIME LAW: R = N/X - Z
  • ANALYTICAL MODEL EXAMPLE OF THE SOLUTION OF AN OPEN MODEL: Model Inputs: V cpu = 121 V disk1 = 70 V disk2 = 50 S cpu = 0.005 S disk1 = 0.030 S disk2 = 0.027 D cpu = 0.605 D disk1 = 2.1 D disk2 = 1.35  = 0.3 jobs/sec
  • Solution Of An Open Model Model Inputs: V cpu = 121 V disk1 = 70 V disk2 = 50 S cpu = 0.005 S disk1 = 0.030 S disk2 = 0.027 D cpu = 0.605 D disk1 = 2.1 D disk2 = 1.35  = 0.3 jobs/sec Model Outputs:  = 1 / D max = 1 / 2.1 = 0.476 jobs/sec X cpu (0.3) =  V cpu = (0.3)(121) = 36.3 visits/sec U cpu (0.3) =  D cpu = (0.3)(0.605) = 0.182 R cpu (0.3) = D cpu / ( 1 – U cpu (0.3) ) = 0.605 / 0.818 = 0.740 secs Q cpu (0.3) = U cpu (0.3) / ( 1 – U cpu (0.3) ) = 0.182 / 0.818 = 0.222 jobs R(0.3) = R cpu (0.3) + R disk1 (0.3) + R disk2 (0.3) = 0.740 + 5.676 + 2.269 = 8.685 secs Q(0.3) =  R(  ) = (0.3) (8.685) = 2.606 jobs EXAMPLE:
  • SUMMARY OF PERFORMANCE METRICS T is the length of TIME we observed the system. A is the number of request ARRIVALS observed. C is the number of request DEPARTURES observed. W is the ACCUMULATED TIME for all requests within the system – time spent both waiting for and using resources. B is the length of time that the resource was observed to be BUSY. Z is the think time of a terminal user. V k the visit ratio, is the number of times device k is visited per transaction. Arrival Rate Y = A / T Throughput (Departure Rate) X = C / T Utilization U = B / T Service Requirement S = B / C Requests in system N = W / T Residence time R = W / C UTILIZATION LAW U = X S LITTLE'S LAW N = X R RESPONSE TIME LAW R = N/X – Z THE FORCED FLOW LAW: X k = V k X
  • OPERATIONAL LAWS
    • Goals:
      • To increase facility using performance metrics.
      • To be able to calculate limits or boundaries on performance metrics.
      • To be able to do &quot;back of the envelope&quot; calculations.
    UTILIZATION LAW U = X S LITTLE'S LAW N = X R RESPONSE TIME LAW R = N/X – Z THE FORCED FLOW LAW: X k = V k X
  • OPERATIONAL LAWS
    • BOUNDARY VALUES:
    • The goal here is to make estimations of the outside limits of a performance parameter. We do this by selective &quot;blind&quot; application of our simple laws to complex systems; our results give upper and lower bounds, not an exact answer.
    • Example:
      • An editor program wants to read 100 disk pages into memory. The resources required for each disk read are:
      • 5 milliseconds of CPU Processor time.
      • 4 milliseconds of Controller Processor time.
      • 2 milliseconds of SCSI Handshaking time.
      • 10 milliseconds of DISK seek/rotation/transfer time.
    • What is the bottlenecking device? Warning!! This is a trick question!
    • What is the worst throughput for this system, assuming the editor single threads the reads - waiting until each read completes before starting the next?
    • What is the best throughput for this system? How long will it take to read in the 100 disk blocks?
  • OPERATIONAL LAWS
    • BOUNDARY VALUES:
    • Now that we know the limits for one user, let's expand this for N users .
    • Example:
    • Multiple users, each with their own copy of the editor, are reading files from the same disk (parameters are the same as the last example.)
    • What is the bottleneck device in this case?
    • Considering only the bottleneck device, what is the best throughput achievable?
    • Now taking into account all devices, what is the BEST throughput N users can achieve ( assuming the requests NEVER get in each other's way?)
    • Now taking into account all devices, what is the WORST throughput N users can achieve ( assuming the requests ALWAYS get in each other's way?)
  • OPERATIONAL LAWS
    • BOUNDARY VALUES:
    • We define DEMAND as the amount of a resource used for a transaction; this is just the number of visits to that resource during one transaction, and the service time used at each of those visits:
        • D k = V k * S k
        • D total = D =  D k
        • D max = D bottleneck
    • SUMMARY:
    • Best throughput overall: X = 1 / D Bottleneck
        • Best throughput for N users: X = N / D
        • Worst throughput for N users: X = N / ( N * D ) = 1 / D
  • OPERATIONAL LAWS Exercise: Consider an interactive system with a CPU and two disks. The following measurement data was obtained by measuring the system: ----------- Observation Data ---------------- Observation interval 30 minutes Active terminals 30 Think time 12 seconds Completed transactions 1,800 Fast disk accesses 36,000 Slow disk accesses 14,400 CPU busy 1,080 seconds Fast disk busy 400 seconds Slow disk busy 600 seconds Determine the visit counts V k , service times per visit S k , and service demands D k at each center. Data At Centers
  • OPERATIONAL LAWS Exercise: ----------- Observation Data ---------------- Observation interval 30 minutes Active terminals 30 Think time 12 seconds Completed transactions 1,800 Fast disk accesses 36,000 Slow disk accesses 14,400 CPU busy 1,080 seconds Fast disk busy 400 seconds Slow disk busy 600 seconds Give optimistic and pessimistic asymptotic bounds on throughput and response time for 10 and 40 active terminals. Bounds on Throughput and Response Time
  • OPERATIONAL LAWS Exercise: ----------- Observation Data ---------------- Observation interval 30 minutes Active terminals 30 Think time 12 seconds Completed transactions 1,800 Fast disk accesses 36,000 Slow disk accesses 14,400 CPU busy 1,080 seconds Fast disk busy 400 seconds Slow disk busy 600 seconds
    • Consider the following changes to the system:
    • Move all files to the fast disk.
    • Replace the slow disk by a second fast disk.
    • Increase the CPU speed by 50% (with the original disks.)
    • Increase the CPU speed by 50% and balance the disk load across 2 fast disks.
    • Rank the efficacy of these changes.
    What changes should be made so response time will not exceed 10 seconds?
  • MEAN VALUE ANALYSIS
    • This is an extension of the Operational Analysis we did before. This is the method that should be used on closed systems.
    • Assumptions:
      • Applies to various service disciplines and service time distributions. (We will only do fixed capacity centers here.)
      • The arrival rate is not dependent on the load.
      • Arrivals are a Poisson process.
    • We will use a technique called Mean Value Analysis that is based on the repetitive application of three equations:
        • 1. Little's Law applied to the whole network:
        • 2. Little's Law applied to each service center:
        • 3. The service center residence time equations:
    A great book on this, now out of print is at http://www.cs.washington.edu/homes/lazowska/qsp/ Menasce, et. al. do this in Section 12.3 of the text.
  • MEAN VALUE ANALYSIS 1. Little's Law applied to the whole network: where X(N) is the system throughput, Z is the think time, and R k (N) the residence time at center k when there are N customers in the network. 2. Little's Law applied to each service center: 3. The service center residence time equations: Delay Centers Queueing Centers where A k (N) is the average number of customers seen at center k when a new customer arrives. The number seen on arrival (in equilibrium) is equal to the average queue length when there is one less customer in the system! So the equations bootstrap!!!
  • MEAN VALUE ANALYSIS //////////////////////////////////////////////////////////////////////////////// // This program does very simple mean value analysis. //////////////////////////////////////////////////////////////////////////////// #define MAX_NUMBER_OF_CENTERS 30 int number_of_customers; int number_of_centers; /* # of centers requested by the user. */ int number_of_customers; /* The population in each class */ float input; double think_time; /* The time spent at the terminal */ double demand[MAX_NUMBER_OF_CENTERS];/* Service demand of a center. */ double q_length[MAX_NUMBER_OF_CENTERS]; double residence_time[MAX_NUMBER_OF_CENTERS+1]; // Residence time at a center. double throughput; double system_response_time; // Prototypes void print_the_numbers(int n); This program is at: http://web.cs.wpi.edu/~jb/perf/Lectures/mva.c
  • MEAN VALUE ANALYSIS main() { char input_string[80]; int center, n; //////////////////////////////////////////////////////////////////////////// // Get the input information from the user. //////////////////////////////////////////////////////////////////////////// printf( &quot;Number of users: &quot;); gets( input_string ); number_of_customers = atoi( input_string ); printf( &quot;Number of centers: &quot;); gets( input_string ); number_of_centers = atoi( input_string ); printf( &quot;Think Time: &quot;); gets( input_string ); sscanf( input_string, &quot;%f&quot;, &input ); think_time = (double)input; for ( center = 0; center < number_of_centers; center++ ) { printf( &quot; Demand for center %d: &quot;, center); gets( input_string ); sscanf( input_string, &quot;%f&quot;, &input ); demand[center] = (double)input; } //////////////////////////////////////////////////////////////////////////// // Now that the network is described, we perform the evaluation. // Begin by initializing to the trivial solution for 0 customers. //////////////////////////////////////////////////////////////////////////// for ( center = 0; center < number_of_centers; center++ ) { q_length[center] = 0; }
  • MEAN VALUE ANALYSIS //////////////////////////////////////////////////////////////////////////// // The algorithm now iterates through each population value n. //////////////////////////////////////////////////////////////////////////// for ( n = 0; n <= number_of_customers; n++ ) { system_response_time = 0; for ( center = 0; center < number_of_centers; center++ ) { residence_time[center] = demand[center] * ( 1.0 + q_length[center] ); system_response_time += residence_time[center]; } //////////////////////////////////////////////////////////////////////////// // Next use Little's Law to compute the system throughput. //////////////////////////////////////////////////////////////////////////// throughput = n / ( think_time + system_response_time ); //////////////////////////////////////////////////////////////////////////// // Finally use Little's Law to compute center queue lengths. //////////////////////////////////////////////////////////////////////////// for ( center = 0; center < number_of_centers; center++ ) { q_length[center] = residence_time[center] * throughput; } print_the_numbers(n); } /* END OF LOOP THROUGH POPULATION */ exit(0); } /* END OF MAIN */
  • MEAN VALUE ANALYSIS //////////////////////////////////////////////////////////////////////////////// // Print the results of each loop //////////////////////////////////////////////////////////////////////////////// void print_the_numbers(int n) { int i; printf( &quot; &quot; ); printf( &quot;Iteration: %d System Response Time: %f Throughput: %f &quot;, n, system_response_time, throughput ); for ( i = 0; i < number_of_centers; i++ ) printf( &quot;Center: %d Residence Time: %f Queue Length: %f &quot;, i, residence_time[i], q_length[i] ); } // END OF PRINT_THE_NUMBERS
  • MEAN VALUE ANALYSIS EXAMPLE OF THE SOLUTION OF A CLOSED MODEL: Model Inputs: V cpu = 121 V disk1 = 70 V disk2 = 50 S cpu = 0.005 S disk1 = 0.030 S disk2 = 0.027 D cpu = 0.605 D disk1 = 2.1 D disk2 = 1.35  = 0.3 jobs/sec The INPUT numbers are the same as were used for the open model. We modify the picture so there are no “arrivals” or “departures”, but instead there are terminals – the jobs stay in the picture (which is what makes it closed.) There are 3 terminal users with average think time of 15 seconds; N = 3, Z = 15 . Model Structure: Terminals CPU Disk1 Disk2
  • Solution Of An Open Model Model Inputs: V cpu = 121 V disk1 = 70 V disk2 = 50 S cpu = 0.005 S disk1 = 0.030 S disk2 = 0.027 D cpu = 0.605 D disk1 = 2.1 D disk2 = 1.35  = 0.3 jobs/sec
    • To start with, we initialize the queue lengths to 0:
    • Q cpu = Q disk1 = Q disk2 = 0
    • Iteration Number 1:
        • R cpu = 0.605 sec R disk1 = 2.1 sec R disk2 = 1.35 sec.
    • X = 1 / ( 15 + 4.055 ) = 0.0525
    • Q cpu = 0.0318 Q disk1 = 0.1102 Q disk2 = 0.0708
    • Iteration Number 2:
        • R cpu = 0.624 sec R disk1 = 2.331 sec R disk2 = 1.446 sec.
    • X = 0.1030
    • Q cpu = 0.0643 Q disk1 = 0.2403 Q disk2 = 0.1490
    EXAMPLE:
  • Solution Of An Open Model Model Inputs: V cpu = 121 V disk1 = 70 V disk2 = 50 S cpu = 0.005 S disk1 = 0.030 S disk2 = 0.027 D cpu = 0.605 D disk1 = 2.1 D disk2 = 1.35  = 0.3 jobs/sec
    • Iteration Number 3:
        • R cpu = 0.644 sec R disk1 = 2.605 sec R disk2 = 1.551 sec.
    • X = 0.1515
    • Q cpu = 0.0976 Q disk1 = 0.3947 Q disk2 = 0.2350
    • From these numbers we can calculate:
        • X(3) = 0.1515
        • R(3) = ( N / X ) - Z = 4.74
    • Q(3) = N - X(3) Z = 0.72
    • There’s a very nice example on Jain, pp 578-9.
    EXAMPLE:
  • CONCLUSION This section has discussed: Queueing Lingo - all the definitions you’ll ever need! Queueing - especially the Single Queue in DETAIL Analytical Models - how to solve a number of complex problems using the equations we already know and love. Works especially well for open models. Operational Laws - drawing conclusions about limits - what to do when we can’t solve the math exactly. Mean Value Analysis - Using the equations in an iterative fashion to solve closed models.