This document summarizes a presentation on maximum likelihood estimation of demands from queue length data in closed queueing networks. It introduces a queueing maximum likelihood estimation (QMLE) approach that can efficiently estimate service demands using only mean queue length observations. The QMLE approach is validated on random models and a commercial application case study, showing errors below 4% and matching observed throughputs. An extension is also presented to handle load-dependent demand scaling.

1. ACM/SPEC ICPE 2016 - March14th, 2016
Weikun Wang (Imperial College London, UK)
Giuliano Casale (Imperial College London, UK)
Ajay Kattepur (TCS Innovation Labs, India)
Manoj Nambiar (TCS Innovation Labs, India)
Maximum Likelihood Estimation of
Closed Queueing Network
Demands from Queue Length Data

2. 2
SPE & Model-Driven Engineering
Platform-Indep.
Model
Architecture
Model
Platform-Specific
Model
Performance
& Reliability Models

3. 3
CQN Characterization
CPU-1
Time
Service
Demand
X X X
CPU-1
Mix 1 Mix 2 Mix 3
 Service demand of a request
 CPU time, bandwidth consumed, …
 Multi-threaded software
 e.g., web servers

4. 4
Drawback of Existing methods
 Utilization based approaches
 Regression based on utilization and throughput
 Issues: collinearities, load-dependence, outliers,
utilization unreliable/unavailable, …

5. 5
 State observations
 Dataset ( points):
 CQN State:
Queue length samples

6. 6
 State observations
 Dataset ( points):
 CQN State:
Queue length samples

7. 7
 Assume product-form state probabilities
 Computationally challenging to evaluate
 Maximum likelihood estimation?
 Infer demands with the probability
Queue length samples
Service demand
Normalizing constant
Queue length

8. 8
 Maximum likelihood estimation (MLE)
 Problem with direct computation
 Evaluation of for each observation
 Slow due to the need for computing
 Very small probabilities when L is large
 Any other solution?
Maximum Likelihood Estimation
parameter space Likelihood

9. 9
 A necessary condition for a point inside to
be a MLE is that
 How to find the MLE?
 Change the value of , until the mean queue
length predicted with MVA match
 Fixed point iteration or an optimization program
Maximum Likelihood Estimation
observed mean
queue length
theoretical mean
queue length
Only mean
queue length
is required!

10. 10
Confidence Intervals
 Assume the MLE to be asymptotically normal
 Confidence intervals for the MLE demands
 is the Fisher Information matrix
 is the Hessian matrix
 works with mean queue length only!
 Obtained by using standard MVA, no probabilities!

11. 11
QMLE Approximation
 Exact MLE can be found by direct search
 Fixed-point iteration tends to be effective
 A simple approximation of the MLE:
 Consider the demand vector where
 Then it must be
observed mean
queue length

12. 12
Validation- Existing methods
 CI: Complete Information
[J.F. Perez et al., IEEE Trans. Sw. Eng.’15]
 Full knowledge of sample path
 Baseline approach
 ERPS: Extended Regression for Processor Sharing
[J.F. Perez et al., IEEE Trans. Sw. Eng.’15]
 Based on mean response time and arrival queue
 GQL: Gibbs Sampling for Queue Lengths
[W. Wang et al., Accepted to appear in ACM TOMACS]
 Gibbs sampling based on queue length samples
 Many iterations until convergence

13. 13
Validation
 ≈20000 random models
 Randomized number of stations, classes, jobs
 Focus on QMLE instead of exact analysis
 Results
 All the algorithms: below 10%
 QMLE has less than 4% error
 Confidence interval validated
Number of
observations

14. 14
1E-04
1E-03
1E-02
1E-01
1E+00
1E+01
1E+02
1E+03
1E+04
CI ERPS GQL QMLE
Execution timeExecutiontime(s)

15. 15
 Mean demand varies under different load
 Real world system behavior
 e.g. multi-core servers
Load-dependent (LD) extension

16. 16
 A scaling factor function
 load-independent :
 Product-form still holds
 MLE
Load-dependent (LD) extension
new term

17. 17
 Directly computation is infeasible
 A necessary condition for a point inside
to be a MLE is that
and
 Works with marginal probability only!
MLE characterization
Empirical marginal
queue length probability
Theoretical marginal
queue length probability

18. 18
 How to find the MLE?
 Solve by optimization program
 Confidence intervals
 Hessian matrix can still be derived
 Computation requires marginal probabilities and
mean queue length only
 Drawback
 Computationally expensive because of LD-MVA
MLE characterization

19. 19
Validation
 Random models validation
 2 stations, 2 classes, 8 jobs, different think time
 MATLAB fmincon solver
 Compare the estimated against exact ones
 Considered scaling factors
 : resembles multi-core feature
– number of CPUs in queueing station i.

20. 20
0
5
10
15
20
25
30
35
40
5000 10000 50000 100000
u
Estimation error on γ (scaling factors)
Erroronγ(%)
Number of observed samples (L)

21. 21
0.0E+00
5.0E+03
1.0E+04
1.5E+04
2.0E+04
2.5E+04
3.0E+04
5000 10000 50000 10000
Execution timeExecutiontime(s)
Number of observed samples (L)
Progress: We found a new approximation
method for efficiently evaluating marginal
probabilities, which reduces the execution
time to < 20s on average!

22. 22
Case Study (MyBatis JPetStore)
 3-tier commercial application
 Transactions grouped in R=1 class
 5 GB user data
User 1
Worker Database
Web/Application server
Workload
Generator
Dispatcher
Database server

23. 23
Observed performance matching
 Exact demand unknown
 Estimated demands using QMLE
 Validate observed throughput with estimated demands
0
5
10
15
20
25
0.1 0.5 1 5 All
ERPS
QMLE
Erroronthroughput(%)
Think time (s)

24. 24
Conclusion
 Demand estimation from queue length
 Efficient
 Confidence interval characterization
 Load-dependent extension
 Ongoing work
 Accelerate the load-dependent estimation
 More experimental evaluations
Funded by FP7 MODAClouds, H2020 DICE, EPSRC OptiMAM

25. 25
weikun.wang11@imperial.ac.uk
Thanks!