1. Hooman Homayounfard, Paul Kennedy & Robin Braun
Faculty of Engineering & Information Technology
University of Technology, Sydney
{hooman.homayounfard, paul.kennedy, Robin.Braun}@uts.edu.au
NARGES: Loss prediction model based on
symbolic delay forecasting
27 September 2013

2. The 17th Pacific-Asia Conference on Knowledge Discovery and Data Mining
(PAKDD2013)
Agenda
 Introduction
 Background
 NARGES Model
 Model Evaluation
 Results
 Conclusions
227 September 2013

3. The 17th Pacific-Asia Conference on Knowledge Discovery and Data Mining
(PAKDD2013)
Introduction: Internet
FISH Problem
327 September 2013
Can we risk by sending valuable data stream to a
highly congested “Hi-Performance” link?
AS 2AS 1 Internet Cloud

4. The 17th Pacific-Asia Conference on Knowledge Discovery and Data Mining
(PAKDD2013)
Introduction
 Quality of Service (QoS) time-series
associated to a link within the Internet
 The “time cost” of numerical time-series
analysis methods for “trivial forecasting
accuracy” is prohibitive.
 Researchers suggest a transition from
computing with numbers to the manipulation
of perceptions in the form of linguistic
variables (Zadeh, 2001; Keogh et al., 2005; Batyrshin &
Sheremetov, 2008).
427 September 2013

5. The 17th Pacific-Asia Conference on Knowledge Discovery and Data Mining
(PAKDD2013)
Introduction: Quality of
Service (QoS)
 Related network parameters that have impact on
the QoS of the audio/video applications
Delay
Jitter
Packet-loss
 There is correlation between delay (and jitter)
values and the probability of packet-loss
occurrence in a link. (Roychoudhuri 2007)
527 September 2013

6. The 17th Pacific-Asia Conference on Knowledge Discovery and Data Mining
(PAKDD2013)
 “Computing with words as a transformation of an initial data
set (IDS) into a terminal dataset (TDS)” (Zadeh ,1999 )
 Computing with Words (CW): p and q are prepositions
stated in human linguistic terms
Background (I)
6
CWIDS TDS
Initial data set: (p) Terminal data set: (q)
27 September 2013

7. The 17th Pacific-Asia Conference on Knowledge Discovery and Data Mining
(PAKDD2013)
Background (II)
7
 Let X, Y be independent random variables
taking values in a finite set of linguistic
terms V = {v1,v2,v3,…,vn} with probabilities
p1,…, pn and q1,…, qn, respectively.
27 September 2013

8. The 17th Pacific-Asia Conference on Knowledge Discovery and Data Mining
(PAKDD2013)
NARGES Model Overview
827 September 2013
Forecasted
Delay
Forecasted
Jitter
Forecasted
Delay
Forecasted
Delay & Jitters
(t)
Packet-Loss Time Series Data Network (used for training MLP ANN)
Measured
QoS Time Series
(t-1 & t-2)
Forecasting
Module
(HDAX)
Pattern
Classification &
Approximation
Pattern Database/Look-up Table
of triplet i-j-k Trend Patterns
and their observed frequencies
Pattern
Frequencies
Retrieval for
Approximation
Storing
Patterns
& Frequencies
PDB Predicted
Packet-Loss
(t+1)
Delayed
Incoming
QoS Data
(Delay, Jitter,
Packet-loss)
MLP ANN
(Predictive
Module)
Predicted
Packet-Loss
Forecasted
Jitter
Measured
Jitter
Measured
Delay
 NARGES cosist of two subcomponent:
 Forecasting Module (HDAX)
 Predictive Module (MLP)

9. Forecasting Module (HDAX)
Forecasted
Delay
Forecasted
Delay & Jitters
(t)
Measured
QoS Time Series
(t-1 & t-2)
Forecasting
Module
(HDAX)
Pattern
Classification &
Approximation
Pattern Database/Look-up Table
of triplet i-j-k Trend Patterns
and their observed frequencies
Pattern
Frequencies
Retrieval for
Approximation
Storing
Patterns
& Frequencies
PDB
Delayed
Incoming
QoS Data
(Delay, Jitter,
Packet-loss)
Forecasted
Jitter
Measured
Jitter
Measured
Delay
927 September 2013

10. The 17th Pacific-Asia Conference on Knowledge Discovery and Data Mining
(PAKDD2013)
HDAX: Deterministic
Mapping Function (DMF)
Basic Patterns are described in terms of
linguistic variables by Deterministic mapping
Function (DMF) like
Alphabet = {SI, I, P, D, SD, OUT}
 DMF: The scale of time-series trends
27 September 2013 10

11. The 17th Pacific-Asia Conference on Knowledge Discovery and Data Mining
(PAKDD2013)
HDAX: Basic & Composite
Patterns
 Pattern Definition
a. Basic Pattern
b. Composite Pattern
27 September 2013

12. The 17th Pacific-Asia Conference on Knowledge Discovery and Data Mining
(PAKDD2013)
 Transition from Number to Linguistic
Values
HDAX: Transition from
Number to Linguistic Values
12
SI
SI SD
SDSI
t
Y
Time
Values
f : a numerical (crisp) function
f *: approximation of
f
If Time is t then Y is Yt
Yt is a Linguistic value assigned to a Y numerical value at time t
D D P P
SI
27 September 2013

13. The 17th Pacific-Asia Conference on Knowledge Discovery and Data Mining
(PAKDD2013)
6000 -
-
5000 -
-
4000 -
-
3000 -
-
2000 -
-
1000 -
-
Y-Axis
Time
SD(4)
I (1) D (3) P (0)
I (1)
P (0) SI (2) I (1)
SI (2)
SD (4)
SI (2)
I (1) D (3)
Time-SeriesValues
End-to-endDelay/Jitter
SD (4)
I – D – P
P – SI – I
Pattern Database
(Runtime: Lookup Table)
.
.
.
.
.
.
.
.
.
P – P – P
SD – SD – SD
.
.
.
Y-Axis
yt-2
yt-1
yt
yt-3
HDAX: Training & Simulation
13
Previous(i)
Current(j)
Next(k)
• Training time: The i − j − k patterns are counted to populate the
corresponding i − j − k frequency entries in the pattern database.
• Simulation time:
•The frequencies stored in Pattern Database (Look-up Table) is used for finding
the most frequent pattern having previous two time-series trends.
• This estimated trend is used to approximate the yt values using Euler method.
27 September 2013

14. The 17th Pacific-Asia Conference on Knowledge Discovery and Data Mining
(PAKDD2013)
Predictive Module (MLP)
1427 September 2013
Forecasted
Jitter
Forecasted
Delay
Forecasted
Delay & Jitters
(t)
Predicted
Packet-Loss
(t+1)
MLP ANN
(Predictive
Module)
Predicted
Packet-Loss
Measured
QoS Time Series
(t-1 & t-2)
Delayed
Incoming
QoS Data
(Delay, Jitter,
Packet-loss)
Measured Packet-Loss Time Series
(used for training MLP ANN)

15. The 17th Pacific-Asia Conference on Knowledge Discovery and Data Mining
(PAKDD2013)
 Parameter settings is done based on the
optimum MLP results of
 Loss Function (NRMSE)
 Performance (speed)
 MLP Training
The Levenberg-Marquardt optimisation process is
based upon Bayesian regularization that
minimises:
 MSE, and
 weights,
MLP: Parameter Setting
& Training
1527 September 2013

16. The 17th Pacific-Asia Conference on Knowledge Discovery and Data Mining
(PAKDD2013)
Data Sources (I)
 Datasets:
 Archives are provided by Napoli University
“Federico II” generated by D-ITG Traffic Generator
http://www.grid.unina.it/Traffic/Traces/dtraces
(36 Datasets)
 Network logs simulated on OPNET Modeller
http://www.opnet.com/solutions/network_rd/modeler
.html
(10 Datasets)
 Data used for:
 Training (25%)
 Simulation (75%).
1627 September 2013

17. The 17th Pacific-Asia Conference on Knowledge Discovery and Data Mining
(PAKDD2013)
Model Evaluation
 Evaluation Benchmark: Autoregressive Moving
Average (ARMA)
 Measurements
 Error Measurement (Loss Function):
Normalised Root Mean Square Error
(NRMSE)
Speed Measurement
 Cross-Correlation Function (CCF)
 Non-parametric Holm’s test used to interpret the
results. We Tested the similarity of algorithms for
accuracy, speed and cross-correlation (CCF) over Delay,
Jitter and Packet-Loss
1727 September 2013

18. The 17th Pacific-Asia Conference on Knowledge Discovery and Data Mining
(PAKDD2013)
Results: Experiments
D-ITG Datasets
 Holm Table for α = 0.05. Models printed in bold are
statistically significantly better.
1827 September 2013

19. The 17th Pacific-Asia Conference on Knowledge Discovery and Data Mining
(PAKDD2013)
Results: Experiments
OPNET Datasets
 Holm Table for = 0.05. Models printed in bold are
statistically significantly better.
1927 September 2013

20. The 17th Pacific-Asia Conference on Knowledge Discovery and Data Mining
(PAKDD2013)
Conclusions & Future Work
 Conclusions
 NARGES results were statistically shown to have
significant correlation and precision compared to the
measured values on a link.
 The results demonstrated the quality and precision
improvement of NARGES compared to ARMA model.
 Future Work
 Implementing the model within a co-simulation
environment for online model evaluation.
 Info. on: www-staff.it.uts.edu.au/~hoomanhm
2027 September 2013

21. The 17th Pacific-Asia Conference on Knowledge Discovery and Data Mining
(PAKDD2013)
Thanks for your attention
Hooman Homayounfard
Hooman.Homayounfard@uts.edu.au
27 September 2013 21

22. The 17th Pacific-Asia Conference on Knowledge Discovery and Data Mining
(PAKDD2013)
References
 Batyrshin, I. Z. & Sheremetov, L. B. (2008), `Perception-based approach
to time series data mining', Applied Soft Computing Journal 8(3),1211-
1221.
 Biaz, S. & Vaidya, N. H. (1998), Distinguishing congestion losses from
wireless transmission losses: A negative result, in `7th Int. Conf.
Computer Communications and Networks', Lafayette, LA, pp. 722-731.
Proceedings of the Seventh International Conference on Computer
Communications and Networks (IC3N).
 Botta, A., Dainotti, A. & Pescap, A. (2007), Multi-protocol and multi-
platform traffic generation and measurement, in `IEEE INFOCOM 2007',
Anchor-age, Alaska, USA.
 Box, G. E. P., Jenkins, G. M. & Reinsel, G. C. (2008), Time series
analysis: forecasting and control, Wiley series in probability and
statistics, 4th edn, John Wiley, Hoboken, N.J.
 Brakmo, L. S., O'Malley, S. W. & Peterson, L. L. (1994), `TCP Vegas:
New techniques for congestion detection and avoidance', ACM
SIGCOMM Computer Communication Review 24(4), 24-35.
 Bui, V., Zhu, W., Pescape, A. & Botta, A. (2007), Long horizon end-to-
end delay forecasts: A multi-step-ahead hybrid approach, in `12th IEEE
Symposium on Computers and Communications, 2007. ISCC 2007',
IEEE, pp. 825-832.
27 September 2013 22

23. The 17th Pacific-Asia Conference on Knowledge Discovery and Data Mining
(PAKDD2013)
References
 Carter, R. L. & Crovella, M. E. (1996), Measuring bottleneck link speed
in packet-switched networks, Vol. 27, 28 of Performance evaluation,
Boston University, Boston, MA, USA.
 Chatfield, C. (2001), Time-series forecasting, Chap-man & Hall.
 Debenham, J. & Simoff, S. (2007), Informed agents: Integrating data
mining and agency, in C. Boukis, L. Pnevmatikakis & L. Polymenakos,
eds, `International Federation for Information Processing-IFIP', Vol. 247,
Springer-Verlag, Boston, pp. 165-173.
 Dovrolis, C., Ramanathan, P. & Moore, D. (2001), What do packet
dispersion techniques measure?, in `INFOCOM 2001. Twentieth Annual
Joint Conference of the IEEE Computer and Communications Societies.
Proceedings. IEEE', Vol. 2, pp. 905-914.
 Handley, M. (1997), An examination of MBone performance, Technical
report, ISI Research Report ISI/RR-97.
 Hannan, E. & Rissanen, J. (1982), `Recursive estimation of mixed
autoregressive-moving average order', Biometrika 69(1), 81-94.
27 September 2013 23

24. The 17th Pacific-Asia Conference on Knowledge Discovery and Data Mining
(PAKDD2013)
References
 Hatonen, K., Klemettinen, M., Mannila, H., Ronkainen, P. & Toivonen, H.
(1996), TASA: Telecommunication alarm sequence analyzer or how to
enjoy faults in your network, in `Network Operations and Management
Symposium, 1996., IEEE', Vol. 2, pp. 520-529.
 Hermanns, O. & Schuba, M. (1996), `Performance investigations of the
IP multicast architecture', Computer Networks and ISDN Systems 28(4),
429-439.
 Jain, M. & Dovrolis, C. (2002), End-to-end available bandwidth:
Measurement methodology, dynamics, and relation with TCP
throughput, in `Proceedings of the 2002 SIGCOMM conference 4', Vol.
32, ACM New York, NY, USA, pp. 295-308.
 Jain, R. (1989), `A delay-based approach for congestion avoidance in
interconnected heterogeneous computer networks', SIGCOMM Comput.
Commun. Rev. 19(5), 56-71. 74686.
 Jiang, W. & Schulzrinne, H. (2000), `Modeling of packet loss and delay
and their effects on real-time multimedia service quality'. ACM Network
and Operating Systems Support for Digital Audio and Video.
 Keogh, E., Lin, J. & Fu, A. (2005), Hot SAX: Efficiently finding the most
unusual time series subsequence, in `Fifth IEEE International
Conference on Data Mining (ICDM'05)', IEEE, Houston, pp. 1-8. Fifth
IEEE International Conference on Data Mining.
27 September 2013 24

25. The 17th Pacific-Asia Conference on Knowledge Discovery and Data Mining
(PAKDD2013)
References
 Lau, H. Y. K. & Woo, S. O. (2007), `An agent-based dynamic routing
strategy for automated material handling systems', International Journal
of Computer Integrated Manufacturing 21(3), 269-288.
 Miloucheva, I., Hofmann, U. & Gutirrez, P. A. A. (2003), Spatio-temporal
QoS pattern analysis in large scale internet environment, in G. Ventre &
R. Canonico, eds, `Interactive Multimedia on Next Generation Networks,
LNCS', Vol. 2899, Springer, Berlin, p. 282. Interactive Multimedia on
Next Generation Networks: First International Work shop on Multimedia
Interactive Protocols and Systems, MIPS 2003, Napoli, Italy, November
2003: Proceedings.
 Moon, S. B. (2000), Measurement and analysis of end-to-end delay and
loss in the Internet, PhD thesis, University of Massachusetts, Amherst.
 Paxson, V. (1998), `Measurements and analysis of end-to-end internet
dynamics', University of California at Berkeley, Berkeley, CA .
 Rankin, J., Christie, G. & Kondratova, I. (2005), Mobile multimodal
solutions for project closeout, in `Proceedings of the CSCE 6th
Construction Specialty Conference,', Toronto, Ontario, pp. 2-4.
 Rocha-Mier, L. E., Sheremetov, L. & Batyrshin, I. (2007), `Intelligent
agents for real time data mining in telecommunications networks',
Lecture Notes in Computer Science 4476, 138.
27 September 2013 25

26. The 17th Pacific-Asia Conference on Knowledge Discovery and Data Mining
(PAKDD2013)
References
 Roychoudhuri, L. & Al-Shaer, E. (2005), `Real-time packet loss
prediction based on end-to-end delay variation', IEEE Trans. Network
Service Manager 2(1).
 Tobe, Y., Tamura, Y., Molano, A., Ghosh, S. & Tokuda, H. (2000),
Achieving moderate fairness for UDP ows by path-status classification,
in `Local Computer Networks, 2000. LCN 2000. Proceedings. 25th
Annual IEEE Conference on', pp. 252-261.
 Wang, Z. & Crowcroft, J. (1991), `A new congestion control scheme:
slow start and search (Tri-S)', SIGCOMM Comput. Commun. Rev. 21(1),
32-43. 116033.
 Weiss, G., Eddy, J., Weiss, S. & Dube, R. (1998), Intelligent
telecommunication technologies, in L. C. Jain, R. Johnson, Y. Takefuji &
L. Zadeh, eds, `Knowledge-Based Intelligent Techniques in Industry',
CRC Press, Boca Raton, pp. 249-275.
 Zadeh, L. A. (2001), `From computing with numbers to computing with
words from manipulation of measurements to manipulation of
perceptions', Annals of the New York Academy of Sciences 929(1), 221-
252.
27 September 2013 26