Design and analysis of a model predictive controller for active queue management


Published on

Model predictive (MP) control as a novel active queue management (AQM) algorithm in dynamic computer networks is proposed. According to the predicted future queue length in the data buffer, early packets at the router are dropped reasonably by the MPAQM controller so that the queue length reaches the desired value with minimal tracking error. The drop probability is obtained by optimizing the network performance. Further, randomized algorithms are applied to analyze the robustness of MPAQM successfully, and also to provide the stability domain of systems with uncertain network parameters. The performances of MPAQM are evaluated through a series of simulations in NS2. The simulation results show that the MPAQM algorithm outperforms RED, PI, and REM algorithms in terms of stability, disturbance rejection, and robustness.

Published in: Business
  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Design and analysis of a model predictive controller for active queue management

  1. 1. ISA Transactions 51 (2012) 120–131 Contents lists available at SciVerse ScienceDirect ISA Transactions journal homepage: Design and analysis of a model predictive controller for active queue management Ping Wang, Hong Chen∗ , Xiaoping Yang, Yan Ma Department of Control Science and Engineering, Jilin University, Campus NanLing, 130025 Changchun, PR China a r t i c l e i n f o Article history: Received 27 March 2010 Received in revised form 11 May 2011 Accepted 25 August 2011 Available online 1 October 2011 Keywords: Network congestion control Active queue management Model predictive control Stability Randomized algorithms a b s t r a c t Model predictive (MP) control as a novel active queue management (AQM) algorithm in dynamic computer networks is proposed. According to the predicted future queue length in the data buffer, early packets at the router are dropped reasonably by the MPAQM controller so that the queue length reaches the desired value with minimal tracking error. The drop probability is obtained by optimizing the network performance. Further, randomized algorithms are applied to analyze the robustness of MPAQM successfully, and also to provide the stability domain of systems with uncertain network parameters. The performances of MPAQM are evaluated through a series of simulations in NS2. The simulation results show that the MPAQM algorithm outperforms RED, PI, and REM algorithms in terms of stability, disturbance rejection, and robustness. © 2011 ISA. Published by Elsevier Ltd. All rights reserved. 1. Introduction With the rapid development of the Internet, network con- gestion is occurring more frequently. Active queue management (AQM) has become an attractive control strategy for congestion avoidance [1,2]. The random early detection (RED) [3] method, popular at the beginning of studying AQM controllers, however, is too sensitive to parameter configuration. A deep insight of the system dynamics is helpful to find new control algorithms. To this end, a fluid-based dynamic model of the transmission control protocol (TCP) was developed by using stochastic theory in [4]. Based on this TCP model, the fundamentals of control the- ory have been used to analyze and develop new AQM schemes. Proportional–integral (PI) controllers [5] and some advanced pro- portional–integral–derivative (PID) controllers [6] have been pro- posed to improve the performance of TCP/AQM systems. Although these controllers enhance the performance of network systems in wide network scenarios, the control parameters configured in par- ticular network scenarios are not able to be adjusted. Model pre- dictive control (MPC) [7–9] predicts the system dynamics in the future and then determines the optimal control signal during each sampling time. Generalized predictive control (GPC), belonging to the MPC family, has been initially used for network congestion con- trol [10]. Predictive function control (PFC), which is one of the MPC ∗ Corresponding author. Tel.: +86 431 85094831; fax: +86 431 85095243. E-mail addresses: (P. Wang), (H. Chen), (X. Yang), (Y. Ma). variants, has also been proposed as an AQM method in [11]. The time delay in the forward/backward channel can actively be com- pensated using a model predictor. Based on control theory, the stability is first considered in the system design. Stability analysis plays an important role in selecting the parameters of AQM controllers. The authors of [5] showed that the RED method cannot guarantee stability when network delays are taken into account. A linearized stability analysis based on the simplified model used in [4] has been developed by taking a proportional controller as the AQM strategy in [12]. The necessary and sufficient conditions of asymptotic stability of the linearized models are obtained. In addition, the stability of congestion control of networks with large round-trip communication delays was addressed in [13]. The authors of [10] analyzed the nominal stability of GPC as an AQM algorithm. In fact, the robustness of the congestion control algorithm is still not well addressed under a dynamic network environment. In the process of improving AQM algorithms, the objective is to establish stability in a parameter space taken as large as possible. A PI controller [5] has stronger robustness than RED in buffer queue regulation. However, it performs poorly when round-trip time (RTT) and load variations occur in real networks [14]. In most cases, the parameters of the linearized model are nonlinearly related, so local stability and robustness with respect to parameter variations cannot be accomplished analytically. The study reported in [15] resorted to non-analytic tools, in which randomized algorithms were used to analyze the stability and robustness of the network system. The study of randomized algorithms has aroused great interest in the systems and control community [16]. 0019-0578/$ – see front matter © 2011 ISA. Published by Elsevier Ltd. All rights reserved. doi:10.1016/j.isatra.2011.08.006
  2. 2. P. Wang et al. / ISA Transactions 51 (2012) 120–131 121 They provide extensive insight into local stability of nonlinear control algorithms. Furthermore, one can obtain accurate results on stability margins. These randomized algorithms are efficient and of low complexity. The main contributions of this paper are as follows. First, we propose a new congestion control algorithm based on MPC, called MPAQM. In this scheme, the queue length is predicted based on the extended TCP/AQM system model and the state estimator. And the network performance index is converted to the objective function of optimal control. The drop probability in the bottleneck link is obtained by solving the optimization problem. Second, properties of the closed-loop system controlled with MPAQM are presented. Furthermore, the margins of robust stability are given by randomized algorithms under uncertain network parameters. Finally, extensive simulations are conducted to demonstrate the effectiveness of MPAQM. The simulation results show that the MPAQM algorithm achieves shorter response time, smaller overshoot, and better stability than RED, PI, and REM. MPAQM also has better responsiveness and robustness. The paper is organized as follows. In Section 2, the fluid flow model and the extended model of the network system are discussed. In Section 3, the MPAQM scheme and some guidelines for parameter settings are explained. Stability and robustness analyses based on randomized algorithms are presented in Section 4. The simulation results of the performance of MPAQM are illustrated by using the network simulator 2 (NS2) in Section 5. Finally, we present our conclusions in Section 6. 2. Modeling of TCP/AQM In this section, we discuss the model of a TCP/AQM intercon- nection system, and the extended system model considering input delay. 2.1. Fluid flow model Consider a single link of capacity C shared by N TCP Reno sources. Ignoring the TCP timeout mechanism, a fluid model of TCP behavior is described by the following coupled and nonlinear differential equations [6]:    ˙w(t) = w(t − R(t)) w(t)R(t − R(t)) (1 − p(t − R(t))) − w(t)w(t − R(t)) 2R(t − R(t)) p(t − R(t)), ˙q(t) = −C + N w(t) R(t) , (1) where w denotes the average TCP window size (packets), q the average queue length (packets), C the link capacity (packets/s), R(t) = q(t)/C + Tp the transmission RTT (s), Tp the propagation delay (s), N the number of TCP sessions, and p the drop probability of a packet, respectively. In [5], it is assumed that (w(t−R(t))/w(t))(1−p(t−R(t))) ≈ 1 with a dropping probability p(·) which is close to zero, so that its additive increase multiplicative decrease (AIMD) model is ˙w(t) = 1 R(t) − w(t)w(t − R(t)) 2R(t − R(t)) p(t − R(t)). (2) We now approximate the dynamic model (1) by small-signal linearization about an operating point. Take (w, q) as the state, p as the input, and q as the output. For a given desired queue length q0, we can derive the following operating point (w0, q0, p0) of model (1): w2 0p0 = 2(1 − p0), w0 = R0C N , R0 = q0 C + Tp. (3) See Appendix for further details. Ignoring the dependence of the time-delay argument t −R(t) on the queue length, we linearize (1) at the operating point to obtain δ¨q(t) = A1δq(t) + A2δ˙q(t) + B1δp(t − R0), (4) where δq = q − q0, δp = p − p0, δ¨q = ¨q, δ˙q = ˙q, A1 = − 2CN R0(2N2+C2R2 0 ) , A2 = − 2CNR0+2N2+C2R2 0 R0(2N2+C2R2 0 ) , B1 = − 2N2+C2R2 0 2R2 0 N . The details of the derivation can be found in [6]. The AQM control strategy introduced in this paper is based on model (4). 2.2. Extended system model Model (4) can be represented as the following state-space model: ˙x(t) = Ax(t) + Bu(t − R0), (5a) y(t) = Csx(t), (5b) where x = [ δq δ˙q ] , A = [ 0 1 A1 A2 ] , B = [ 0 B1 ] , u = δp, Cs =  1 0  . By discretizing (5) at the sampling instants kTs [17], the discrete-time model is consequently given as x(k + 1) = Adx(k) + Bd1u(k − τ) + Bd2u(k − τ − 1), y(k) = Cdx(k), (6) where, Ad = eATs , Bd1 =  R0−τTs 0 eAs Bds, Bd2 =  Ts R0−τTs eAs Bds, Cd = Cs, τ =  R0 Ts  (  R0 Ts  rounds R0 Ts to the nearest integer less than or equal to R0 Ts ). To guarantee regulation with zero steady-state error for the constant set-point, let the control variable u be the following discrete-time integrator: u(k) = u(k − 1) + u(k). (7) Because model (6) highlights explicitly the presence of the system time delay, we define the state vector X(k) and matrices ˜Ad, ˜Bd, ˜Cd as follows [18]: X(k) =       x(k) u(k − 1) u(k − 2) ... u(k − τ − 1)       , ˜Ad =         Ad 0 0 · · · Bd1 Bd2 0 1 0 · · · 0 0 0 1 0 · · · 0 0 0 0 1 · · · 0 0 ... ... ... ... ... ... 0 0 0 · · · 1 0         , ˜Bd =       0 1 0 ... 0       , ˜Cd =       Cd 0 0 ... 0       T . Hence, system (6) can be extended to X(k + 1) = ˜AdX(k) + ˜Bd u(k), (8a) y(k) = ˜CdX(k). (8b) Till then, the network system dynamics is described by (8), which is the basic model of the MPC controller designed in the next section.
  3. 3. 122 P. Wang et al. / ISA Transactions 51 (2012) 120–131 3. Model predictive algorithm of AQM In this section, a new AQM algorithm, MPAQM, based on MPC is designed. The objective is to stabilize the queue length as a target value for improving the network performance and smoothing out the burst traffic under arbitrary feedback delays. The key idea of MPAQM is described in three steps. First, the state vector of model (8) is estimated. We begin with the estimated state vector, and predict the queue length in the future finite horizon based on model (8). Second, the drop probability is determined by solving the optimization problem in order to achieve faster responses and prevent excessive control effort. Finally, the optimal drop probability is applied to the network system as the feedback control signal. According to the basic principles of predictive control, this process is repeated at each sampling time, which makes queue length reach the desired value quickly. The design process of each part is introduced as follows. 3.1. State estimator design The state vector X(k) of model (8) has to be estimated because the second component in x(k) is not available. Here, the estimator is designed by the state-space method. When the estimated state ˆX(k) and the control input increment u(k) at time k are given, state ˆX(k + 1) is calculated by (8a) ˆX(k + 1) = ˜Ad ˆX(k) + ˜Bd u(k) + Kf (y(k) − ˜Cd ˆX(k)), (9) where Kf is the gain matrix of the estimator. The design criteria for Kf are to guarantee the stability of the estimator and the fast convergence of estimation error. In this study, Kf is based on the reginal pole placement theory and LMI (linear matrix inequality) techniques, such that the distribution of eigenvalues of ˜Ad −Kf ˜Cd is a disk region with center at the origin and radius r0 (r0 < 1). The condition of the reginal pole placement is given as follows. Lemma 1 ([19]). The square matrix A has all its eigenvalues σ(A) ⊂ D(r, o) if and only if there exists a symmetric X > 0 such that [ −rX AX − oX XAT − oX −rX ] < 0, (10) where D(r, o) is a disk with center at o and radius r. From Lemma 1, we derive the following LMI for estimator (9). Corollary 1. For the square matrix ˜Ad − Kf ˜Cd, σ(˜Ad − Kf ˜Cd) ⊂ D(r0, 0) if and only if there exists a symmetric X > 0 and Q , such that [ −r0X ˜AT d X − ˜CT d Q T X ˜Ad − Q ˜Cd −r0X ] < 0, (11) when the solutions of Q and X of LMI (11) exist, Kf = X−1 Q is one of the appropriate gain matrices of estimator (9). Remark 3.1. It should be pointed out that, as long as (˜Cd, ˜Ad) is detectable, the solution of LMI (11) exists for any given r0 < 1, i.e., one can find a Kf through (11) such that the estimator (9) is stable. 3.2. Future queue length prediction According to the principles of predictive control [7], at time k, the coming queue length is predicted on the basis of model (8). Here, Nc is defined as the control horizon, and Np is defined as the prediction horizon; hence, the prediction function of the queue length is as follows: YNp (k + 1|k) = Sx ˆX(k) + Su U(k), (12) where YNp (k + 1|k)     y(k + 1|k) y(k + 2|k) ... y(k + Np|k)     , U(k)     u(k) u(k + 1) ... u(k + Nc − 1)     , Sx =      ˜Cd ˜Ad ˜Cd ˜A2 d ... ˜Cd ˜A Np d      , Su =      ˜Cd ˜Bd 0 0 · · · 0 ˜Cd ˜Ad ˜Bd ˜Cd ˜Bd 0 · · · 0 ... ... ... · · · ... ˜Cd ˜A Np−1 d ˜Bd ˜Cd ˜A Np−2 d ˜Bd · · · · · · ˜Cd ˜A Np−Nc d ˜Bd      . 3.3. Optimization and feedback control As mentioned above, the main control requirement of AQM algorithms is to make the queue length q stabilize near the target value q0 as soon as possible to avoid network congestion. But it is difficult to control the TCP sending window because of the time delay. MPAQM determines the drop probability based on the predicted future queue length YNp (k + 1|k). The reference sequences of the queue length are defined as Re(k + 1) =  r(k + 1) r(k + 2) · · · r(k + Np) T , where r(k + i) = 0, i = 1, 2, . . . , Np (because the origin is the operating point of model (4)). The queue length q can converge quickly to q0 by minimizing J1 = ‖YNp (k + 1|k) − Re(k + 1)‖2 . In addition, network fluctuations should be prevented for guaranteeing the quality of the network service. That is to say, the variation range of drop probability should be limited in AQM algorithms. In MPAQM, this requirement is formulated as minimizing J2 = ‖ U(k)‖2 . Because minimizing J1 and J2 simultaneously is contradictive, the weighting factors for different request are given. The MPAQM algorithm is described by the following optimization problem: min U(k) J(y(k), U(k), Nc , Np), (13a) J(y(k), U(k), Nc , Np) = ‖Γy(YNp (k + 1|k) − Re(k + 1))‖2 + ‖Γu U(k)‖2 , (13b) where Γy and Γu are the weighting matrices. By solving the optimization problem (13), the optimal control sequences at time k are derived: U∗ (k) = (ST u Γ T y ΓySu + Γ T u Γu)−1 ST u Γ T y ΓyEp(k + 1|k), (14) where Ep(k + 1|k) = Re(k + 1) − Sx ˆX(k). The first component of U∗ (k) is effectively used to compute the control signal u(k) according to (7). Hence, the closed-loop control law is defined as u(k) = KmpcEp(k + 1|k), (15) where Kmpc =  1 0 · · · 0  (ST u Γ T y ΓySu + Γ T u Γu)−1 ST u Γ T y Γy.
  4. 4. P. Wang et al. / ISA Transactions 51 (2012) 120–131 123 4. Closed-loop properties In this section, we use the Routh–Hurwitz theorem, a common technique in control theory, to analyze the stability of the MPAQM algorithm, and we use randomized algorithms to analyze the robustness under uncertain network parameters. 4.1. Stability analysis In order to analyze the stability of the estimator-based MPAQM algorithm, we first discuss the stability of the state feedback MPAQM. To do this, it is assumed that all states are measured, i.e., ˆX = X. Then, the feedback control (15) becomes u(k) = Kmpc (Re(k + 1) − SxX(k)) , (16) and the corresponding closed-loop system is given as X(k + 1) =  ˜Ad − ˜BdKmpcSx  X(k) + ˜BdKmpcRe(k + 1). (17) It is then clear that the state feedback MPAQM algorithm is stable if and only if ˜Ad − ˜BdKmpcSx is stable, i.e., all its eigenvalues lie in the unit circle. Now we come back to the estimator-based case and need to obtain the corresponding closed-loop system. The network system is still described by (8a), and the estimator is given by (9). By substituting (15) into (9) and (8a), we have X(k + 1) = ˜AdX(k) − ˜BdKmpcSx ˆX(k) + ˜BdKmpcRe(k + 1), (18a) ˆX(k + 1) = Kf ˜CdX(k) + (˜Ad − Kf ˜Cd − ˜BdKmpcSx)ˆX(k) + ˜BdKmpcRe(k + 1). (18b) Hence the estimator-based closed-loop system is given as [ X(k + 1) ˆX(k + 1) ] = [ ˜Ad −˜BdKmpcSx Kf ˜Cd ˜Ad − Kf ˜Cd − ˜BdKmpcSx ] × [ X(k) ˆX(k) ] + [ BdKmpc BdKmpc ] Re(k + 1). (18c) It is then clear that system (18c) is stable if and only if all the eigenvalues of [ ˜Ad −˜BdKmpcSx Kf ˜Cd ˜Ad − Kf ˜Cd − ˜BdKmpcSx ] (19) lie in the unit circle. By taking a similarity transformation with P = [ I 0 −I I ] , P−1 = [ I 0 I I ] (20) on (19), we have P [ ˜Ad −˜BdKmpcSx Kf ˜Cd ˜Ad − Kf ˜Cd − ˜BdKmpcSx ] P−1 = [ ˜Ad − ˜BdKmpcSx −˜BdKmpcSx 0 ˜Ad − Kf ˜Cd ] , (21) where ˜Ad − ˜BdKmpcSx is the closed-loop system matrix when all states are assumed to be measured, and ˜Ad − Kf ˜Cd is the system matrix of the state estimator. Hence, we arrive at the following separation principle. Theorem 1. The estimator-based MPAQM is stable if and only if the state feedback MPAQM and state estimator are stable, respectively. As an example for analyzing the stability of closed-loop system (18c), we consider a dumbbell topology with a single common bottleneck link of 15 Mb/s capacity (see Fig. 1). The other parameters in model (4) are as follows: the round-trip propagation delay Tp is 50 ms, the TCP flows N is 120, and the mean packet size is 500 bytes. Note that the operating point (w0, p0, q0) in (3) depends Fig. 1. The system network model. on the desired queue length q0, which is usually less than one half the buffer size [13,14]. We set q0 = 200 packets, and obtain the operating point R0 = 0.1033 s, p0 = 0.175. We set Ts = 0.01 s; then τ = 10 is obtained. For a better result of estimation, r0 in LMI (11) is set to be 0.4. By solving LMI (11), we obtain Kf =  2.9317 234.7818 − 0.1642 − 0.1642 − 0.1642 −0.1642 − 0.1642 − 0.1642 − 0.1642 −0.1642 − 0.1642 − 0.1642 − 0.1643 T . The maximum modulus of the eigenvalues of matrix ˜Ad − Kf ˜Cd is 0.2563. For the MPC controller, the weights of the cost function are chosen to be Γu = 2000I15, Γy = I15 (I15 is the unit matrix in the 15 × 15-dimensional space); the prediction and control horizons are set to be Np = 15 and Nc = 15, respectively. The maximum modulus of the eigenvalues of matrix (19) is 0.984, which implies that the closed-loop system (18c) is stable. Remark 4.1. The sampling interval affects the discrete-time model (6) and the calculation burden of the MPAQM algorithm. Among existing AQM schemes, the control mechanism of the PI scheme is most similar to that of MPAQM scheme. In [5], it is advisable to operate the PI controller at 10–20 times the loop bandwidth, and its sampling frequency is 160 Hz. We assume that the MPAQM and PI methods have the same bandwidth. To reduce the computation burden, the sampling frequency in our study is set to be 100 Hz (Ts = 0.01 s). 4.2. Robustness analysis The nominal stability of closed-loop system (18c) has just been analyzed when the network parameters are constant. Actually, the TCP loads are time-varying loads, which directly affects the congestion degree, and R0 is the approximate RTT value in equilibrium. Moreover, the model mismatch problem does exist in model (4), where high-frequency dynamics are not strictly taken into account. So we further discuss the robustness of closed-loop system (18c). Because of the random characteristics of the network, random- ized algorithms are used to analyze the robustness. Considering the uncertainty of parameters N and R0, we derive the following model from model (6): x(k + 1) = Ar dx(k) + Br d1u(k − τ) + Br d2u(k − τ − 1), (22) where Ar d, Br d1, Br d2 are random variables. In order to simplify the derivation, the input delay is assumed to be constant. Following the derivation in Section 2.2, we obtain the corresponding extended system: X(k + 1) = ˜Ar dX(k) + ˜Br d u(k), (23a) y(k) = ˜CdX(k). (23b) By substituting (15) into (23a), we have X(k + 1) = ˜Ar dX(k) − ˜Br dKmpcSx ˆX(k) + ˜Br dKmpcRe(k + 1). (24)
  5. 5. 124 P. Wang et al. / ISA Transactions 51 (2012) 120–131 Fig. 2. The first 800 samples of various two-dimensional uniformly distributed pseudorandom sequences. The corresponding closed-loop system is [ X(k + 1) ˆX(k + 1) ] = [ ˜Ar d −˜Br dKmpcSx Kf ˜Cd ˜Ad − Kf ˜Cd − ˜BdKmpcSx ] × [ X(k) ˆX(k) ] + [ Br dKmpc BdKmpc ] Re(k + 1). (25) Then, we analyze the stability of L = [ ˜Ar d −˜Br dKmpcSx Kf ˜Cd ˜Ad − Kf ˜Cd − ˜BdKmpcSx ] (26) in the probabilistic sense based on Monte Carlo methods. Step 1. The parameters N and R0 are taken to be random, with uniform density functions fN and fR0 . Then, we generate M inde- pendent identically distributed samples according to the density function fN and fR0 : N1 , N2 , . . . , NM , R1 0, R2 0, . . . , RM 0 , respectively. Subsequently, we compute Ar d(Ni , Ri 0), Br d1(Ni , Ri 0), Br d2(Ni , Ri 0) in (22) for i = 1, 2, . . . , M, where we have suppressed the de- pendence on the capacity C. By the derivation from (22) to (25), L(Ni , Ri 0) is computed for the robustness analysis. Step 2. Construct the indicator function I(Ni , Ri 0) =  1 If L(Ni , Ri 0) is Schur 0 otherwise. (27) The estimated probability of stability is readily given by ˆpM = 1 M −M i=1 I(Ni , Ri 0). (28) The estimation ˆpM is usually referred to as the empirical probability. We need to know the samples size M to obtain a ‘‘reliable’’ probabilistic estimation ˆpM . Thus, a Chernoff bound is used. The Chernoff bound [20] states that, for any ϵ ∈ (0, 1) and δ ∈ (0, 1), if M ≥ 1 2ϵ2 ln  2 δ  (29) then, with probability greater than 1 − δ, we have |ˆpM − ptrue| < ϵ, where ptrue denotes the real probability of stability. When the samples size M is determined, another important issue in Monte Carlo methods is how to choose good random number generators. We use a multiplicative congruential generator. The quasi-random sequences generated by the multiplicative congruential method are shown in Fig. 2. We analyze the robustness of the concrete network described in Section 4.1 by Monte Carlo methods. The parameters in (29) are set as follows: δ = 0.001 and ϵ = 0.008. We choose the number of samples as M = 60 000, based on the Chernoff bound. We want Fig. 3. The stability probability with uncertain N and R0. Fig. 4. The stability probability versus capacity. to find the stability domain in the parameter space (N, R0) with the MPAQM algorithm. Fig. 3 plots the stability probability when the uncertainties of N and R0 change from 5% to 65% around the normal values 120 and 0.1033, respectively. Fig. 3 shows that the system is stable with ±55% uncertain N and ±50% uncertain R0. We further discuss the influence of different link capacity C on the system stability. We perform simulations with parameter ranges N ∈ [54, 186], R0 ∈ [0.0517, 0.155] and 20 fixed values of capacity: C = 9.6, 10.2, 10.8, 11.4, . . . , 20.4, 21 Mb/s. The samples are generated by the multiplicative congruential and grid method, respectively. The stability probability is shown in Fig. 4. The simulation result shows that the stability of the system with MPAQM decreases, i.e., the queue length in the buffer of bottleneck link cannot reach the target value of 200 packets, when the capacity C is greater than 15 Mb/s. 5. Simulation results In this section, we evaluate the performance and robustness of the proposed MPAQM algorithm by a number of simulations performed in NS2 [21]. The well-known RED, PI, and REM methods are also simulated for the purpose of comparison.
  6. 6. P. Wang et al. / ISA Transactions 51 (2012) 120–131 125 a b Fig. 5. Queue length and drop probability of MPAQM. Fig. 6. Queue length and drop probability of RED. 5.1. Performance evaluation The performance of MPAQM is compared with that of RED, PI, and REM under FTP flows and mixture flows, respectively. The network topology and the model parameters are given in Section 4.1. Some other parameters used in NS2 are set as follows: the buffer size B is 500 packets, and the window size of each TCP connection is 30 packets. 5.1.1. Under FTP flows only First, we investigate the stability and responsiveness of MPAQM under long-lived FTP flows only. Each sender–receiver pair has TCP connections as cross traffic. In both scenarios, TCP Reno is used as the transport agent. The parameters of MPAQM are set to be the values calculated in Section 4.1. For a fair comparison in some sense, the settings of the parameters for the following AQM schemes are based on their authors’ recommendations and have some refinements for this network topology. The basic parameters of RED (see the notation in [1]) are set at intervaltime = 0.5 s, minth = 100 packets, maxth = 280 packets, maxp = 0.01, and wq = 0.002. For the PI scheme, the control parameters are fixed following the rule design recommended in the original work [5]: a = 4.389 × 10−5 , b = 4.346×10−5 . The settings of the parameters for the REM scheme are based on author recommendations [22]: α = 0.1, φ = 1.002, γ = 0.001, b∗ = 200 packets. Figs. 5–8 show the queue lengths and the drop probability ob- tained with different AQM schemes. It can be seen that the queue lengths in Figs. 5, 7 and 8 stabilize after about 4.6 s, 10.3 s, and 7.8 s, respectively, while the queue length of RED fail to do so (it does at about 230 packets). It should be noted that the RED, PI, and REM methods are designed based on the nominal values of network parameters to regulate the queue length; therefore, the best results are expected in this experiment. Other performance measures are given in Table 1. The variation of queue length in the MPAQM method is low and it is the highest in the PI method. Al- though the bottleneck link utilization of the various AQM meth- ods is rather similar, the drop probability in RED has the highest value and in the MPAQM method it has the lowest value. Through comparing, MPAQM gives less overshoot, lower drop probabil- ity, shorter response time,and better stability than the other methods. 5.1.2. Under mixture flows The unresponsive constants bit rate (CBR) flows are like a constant disturbance. It can influence the control effect of AQM algorithms as a result of queue oscillations or unstable queue evolution. In this simulation, we use a mixture of FTP and CBR flows to simulate a more realistic network scenario. In Fig. 1, the number of FTP flows and CBR flows are 120 and 15, respectively. The Internet protocol used by CBR flows is user data protocol (UDP). The inter-packet gap of a CBR
  7. 7. 126 P. Wang et al. / ISA Transactions 51 (2012) 120–131 Table 1 Performance measures under FTP flows. Average q (packets) STD q (packets) Drop probability (%) Link utilization (%) MPAQM 199.657 28.6048 7.2467 99.9012 RED 233.75 57.72 10.23 99.9891 PI 199.739 63.5286 8.7433 99.4178 REM 205.82 46.27 7.36 99.9015 Table 2 Performance measures under mixture flows. Average q (packets) STD q (packets) Drop probability (%) Drop probability (10–30 s)(%) Link utilization (%) MPAQM 196.848 56.1698 15.8145 24.071 99.5156 RED 252.924 110.3782 12.66 17.32 99.9807 PI 200.508 67.72 18.1347 20.4133 98.5732 REM 197.66 147.725 17.47 21.93 91.0271 Fig. 7. Queue length and drop probability of PI. Fig. 8. Queue length and drop probability of REM. flow is 0.005 s, and the mean packet size is 500 bytes. The CBR flows start at 10 s and stop at 30 s. The parameters for MPAQM, RED, PI, and REM are unchanged from the values used in Section 5.1.1. From Figs. 9–12, we see that MPAQM can robustly stabilize the queue length around 200 packets, while the queue length of PI keeps the peak value for around 8 s. REM requires much longer time to decrease its queue size from the buffer top. The queue length in the RED method fluctuates and is far from the desired value when UDP flows exist. Other performance measures are shown in Table 2. As is shown in this table, the queue length variation in the MPAQM method is less than in the others. The performances of RED and REM degrade noticeably. The variation of queue length in the REM method has the highest value. This leads to the underutilization of resources and poor quality of service. Although the loss rate in the RED method has the lowest value, the average queue length is very far from the desired value. The results also show that the queue evolutions of the MPAQM and PI
  8. 8. P. Wang et al. / ISA Transactions 51 (2012) 120–131 127 Fig. 9. Queue length and drop probability of MPAQM under mixture flows. Fig. 10. Queue length and drop probability of RED under mixture flows. Fig. 11. Queue length and drop probability of PI under mixture flows. schemes are based on higher drop probability, but the performance of MPAQM is superior to that of PI. 5.2. Robustness evaluation The robustness of various AQM methods is evaluated under network uncertainty. According to the analysis in Section 4.2, we conduct a set of simulations to investigate the robustness through the TCP connections from 60 to 180 and the round-trip propagation delay from 30 ms to 80 ms. Details of the network setting are given in Table 3. 5.2.1. Robustness with variables N and Tp The bottleneck link capacity and the parameters of MPAQM are the same as the values used in Section 4.1. The parameters N and Tp are set according to Table 3.
  9. 9. 128 P. Wang et al. / ISA Transactions 51 (2012) 120–131 Fig. 12. Queue length and drop probability of REM under mixture flows. Fig. 13. Queue length and drop probability of MPAQM with variables N and Tp. Fig. 14. Queue length and drop probability of RED with variables N and Tp. Fig. 13 shows that the queue length in the MPAQM algorithm robustly stabilizes around the target value, and the drop probabil- ity increases to some degree, while the number of TCP connections increases. Moreover, MPAQM has a rapid regulating process when new TCP connections start to send packets and old TCP connections stop sending packets. From 40 to 60 s, larger values of the propaga- tion delay are considered together with load variations represent- ing a strong congestion scenario. Notice that, in Fig. 14, RED does not achieve a good queue stabilization from a medium load value of about 150 sessions. In Fig. 15, the PI method shows consistent
  10. 10. P. Wang et al. / ISA Transactions 51 (2012) 120–131 129 Fig. 15. Queue length and drop probability of PI with variables N and Tp. Fig. 16. Queue length and drop probability of REM with variables N and Tp. Table 3 Network setting in robustness evaluation. Number of active TCP sessions N 30 30 30 30 30 30 Propagation delay Tp (ms) 30 40 50 60 70 80 Starting time (s) 0 0 10 20 30 40 Stopping time (s) 100 100 90 80 70 60 variations of the queue length and drop probability. We observe that the PI takes the longer time to settle down the equilibrium point. It can be seen from Fig. 16 that REM shows an acceptable behavior when we vary N from 30 to 150 sessions. But it performs poorly for the reverse process. From these figures, we can find that MPAQM achieves a short response time, good stability, and good robustness. 5.2.2. Robustness with different C Based on the results in Section 5.2.1, the experiments in this subsection aim to investigate the ability of the MPAQM scheme under varying bottleneck link capacities. For sake of brevity, only the performance of the MPAQM controller with different C is shown because other AQM schemes presented worse behavior. Four fixed values of bottleneck link capacity are set: C = 5, 10, 20, 25 Mb/s, respectively. The other parameters of the network and MPAQM algorithm are still the values used in Section 5.2.1. Fig. 17 plots the queue evolution of the MPAQM method under different C, which shows that the queue length fails to stabilize around 200 packets over a period of time when C = 20 Mb/s and C = 25 Mb/s. The stability with 25 Mb/s capacity is worse than that with 20 Mb/s. The simulation results verify the analysis result indicated in Fig. 4. 5.2.3. Multiple bottleneck topology with cross traffic Using the multiple bottleneck network topology depicted in Fig. 18, we study the behavior of different AQM algorithms in the presence of cross traffic. We set 120 FTP flows with senders at the left-hand side and receivers at the right-hand side, and 30 FTP flows with each cross sender–receiver pair. The simulated network has two bottleneck links (link R2–R3, link R4–R5). Because link R2–R3 and link R4–R5 exhibit similar trends, we do not plot the data of link R4–R5. The queues of R2–R3 for different AQMs are depicted in Fig. 19. In Fig. 19, we can see that all the AQM schemes have noticeable oscillations in this case with multiple bottlenecks and cross traffic; yet MPAQM controls the queue length significantly better than others. REM and PI exhibit larger queue oscillation than MPAQM. RED oscillates largely in the congested routers, and frequently results in poor link utilization and continuous packet losses. The queue lengths of the PI and RED schemes are still frequently below the expected value (200 packets). In contrast, MPAQM can regulate the queue length around the expected value and obtain a better control effect than the other schemes.
  11. 11. 130 P. Wang et al. / ISA Transactions 51 (2012) 120–131 Fig. 17. Queue length of MPAQM with different capacity C. Fig. 18. The system network model. 6. Conclusions In this paper, we have proposed a novel AQM scheme based on MPC, i.e., MPAQM. The components of the basic framework are the extended state-space model of the network system, a state estimator based on the state-space method, prediction of the queue length, and an optimal controller for improving the network performance. The stability analysis gives the guidelines on how to select the parameters of the MPAQM algorithm to ensure the stability of the network system. Considering the uncertainty of the network parameters, a robustness analysis based on Monte Carlo methods provides the accurate margins of stability. We conducted extensive simulations and have comprehen- sively compared the performance of MPAQM with RED, PI, and REM. The performance analysis and simulation results illustrate the effectiveness of the MPAQM algorithm, which provides bet- ter performances than RED, PI, and REM. The queue length of MPAQM converges to the desired value more quickly. The dynamic response of the queue length has smaller oscillations in the pres- ence of unresponsive UDP flows or parameter variations. In par- ticular, the queue length dynamics is shown to exhibit good robustness and fast system response under multiple bottleneck link scenarios. Future work will cover the extension of the simulation environment from numerical simulations in NS2 to real network experiments and the explicit consideration of time-domain constraints on the drop probability and queue length. Appendix. Derivation of (3) For a given desired queue length q0, the operating point (w0, q0, p0) of model (1) is defined by ˙w = 0 and ˙q = 0, so that    ˙q(t) = 0 ⇒ R0 = q0 C + Tp, ⇒ −C + N w0 R0 = 0, ˙w(t) = 0 ⇒ 1 − p0 R0 − w2 0 2R0 p0 = 0. (A.1) Then, we get (3).
  12. 12. P. Wang et al. / ISA Transactions 51 (2012) 120–131 131 Fig. 19. Queue evolution under multiple bottleneck topology. References [1] Floyd S, Jacobson V. Random early detection gateways for congestion avoidance. IEEE/ACM Transactions on Networking 1997;4(1):1–22. [2] Clark DD, Fang W. Explicit allocation of best effort packet delivery service. IEEE/ACM Transactions on Networking 1998;6(4):362–73. [3] Branden B, Clark D, Crowcroft J. Recommendations on queue management and congestion avoidance in the internet. In: RFC2309. 1994. [4] Misra V, Gong WB, Towsley D. Fluid-based analysis of a network of AQM routers supporting TCP flows with an application to RED. In: Proceedings of ACM/SIGCOMM. 2000. [5] Hollot CV, Misra V, Towsley D, Gong WB. Analysis and design of controllers for AQM routers supporting TCP flows. IEEE Transactions on Automatic Control 2002;6(47):945–59. [6] Kim KB. Design of feedback controls supporting TCP based on the state-space approach. IEEE Transactions on Automatic Control 2006;7(51):1086–99. [7] Lee JH, Morari M. Model predictive control: past, present and future. Computers and Chemical Engineering 1999;23:667–82. [8] Rawlings JB. Tutorial overview of model predictive control. IEEE Control Systems Magazine 2000;3(20):38–52. [9] Veselý V, Rosinová D, Foltin M. Robust model predictive control design with input constraints. ISA Transactions 2010;49:114–20. [10] Zhang XH, Zou KS, Chen ZQ, Deng ZD. Stability analysis of AQM algorithm based on generalized predictive control. In: Proceedings of the 4th inter- national conference on intelligent computing. Berlin (Heidelberg): Springer- Verlag; 2008. p. 1242–8. [11] Bigdeli V, Haeri M. Predictive functional control for active queue management in congested TCP/IP networks. ISA Transactions 2009;48:107–21. [12] Michies W, Melchor-Aguilar D, Niculescu SI. Stability analysis of some classes of TCP/AQM networks. International Journal of Control 2006;9(79):1136–44. [13] Wang JX, Rong L, Liu YH. Design of a stabilizing AQM controller for large-delay networks based on internal model control. Computer Communications 2008; 31:1911–8. [14] Quet PF, Ozbay H. On the design of AQM supporting TCP flows using robust control theory. IEEE Transactions on Automatic Control 2004;6(49):1031–6. [15] Tempo R. Randomized algorithms for stability and robustness analysis of high- speed communication networks. IEEE Transactions on Neural Networks 2005; 5(16):1229–41. [16] Tempo R, Calafiore G, Dabbene F. Randomized algorithms for analysis and control of uncertain systems. New York: Springer-Verlag; 2005. [17] Nilsson J. Stochastic analysis and control of real-time systems with random time delays. Automatica 1998;1(34):57–64. [18] Clarke DW, Scattolini R. Constrained receding horizon predictive control. Proceedings of IEE 1992;(Part D):347–54. [19] Boyd S, El Ghaoui L, Feron E, Balakishnan V. Linear matrix inequalities in system and control theory. Philadelphia: SIAM; 1994. [20] Hellekalek P. Good random number generators are (not so) easy to find. Mathematics and Computers in Simulation 1998;46(5–6):485–505. [21] USC/ISI, Angeles L., CA. The NS simulator and the documentation. Available from: [22] Athuraliya S, Low S, Li V, Yin Q. Rem: active queue management. IEEE Network Magazine 2001;15:48–53.