Performance Management of IT Service Processes Using a Mashup-based Approach - Thesis presentation Hypothesis: The employment of mashups enhances the performance of human-centered ITSM processes

1. PERFORMANCE MANAGEMENT OF IT
SERVICE PROCESSES USING A
MASHUP-BASED APPROACH
Carlos Raniery Paula dos Santos
Supervisor: Lisandro Zambenedetti Granville
Thesis defense – June13rd, 2013

2. Outline
• Introduction
• Boundaries of the Investigation
• Productivity
• Reliability
• Evaluation
• Conclusions and Future Work
2

3. INTRODUCTION

4. • The increasing complexity and importance of IT
environments have been leading researchers to pay
more attention to the ITSM area
• Several management frameworks have then been
conceived to help IT service providers towards a
coherent service management experience
1
4
INTRODUCTION
CONTEXTUALIZATION

5. • In a service management operation the presence of
humans in the critical path for performing work
degrades performance and quality of services
• Due to its unpredictable nature, enforcing and
obtaining tight performance bounds in a human-
staffed organization is burdensome task
5
1INTRODUCTION
PROBLEM STATEMENT

6. • Automation of IT operations has been of attention
for the last two decades
– Ideally, a system should be self-managed without any
human intervention
• May not be feasible in some cases because of
additional effort to deploy and maintain the
automation infrastructure
– Specially when processes are constantly changing, are
too complex to be automated, or have a limited lifetime
6
1INTRODUCTION
CURRENT SOLUTIONS

7. • Mashups are Web applications composed from pre-
existing Web resources (e.g., interactive maps, Web
services, traditional HTML pages)
• Mashups can be composed by users with limited or
no programming skills, to create their own Web
applications
– Basic operators are employed to hide, from mashup end-
users, technical details from the composition
7
1INTRODUCTION
MASHUPS

8. 8
1INTRODUCTION
MASHUPS

9. 9
Q I. What are the major causes for a poor performance in the execution of IT Service
Management processes?
Hypothesis: The employment of mashups enhances the performance of
human-centered ITSM processes
1INTRODUCTION
HYPOTHESIS AND FUNDAMENTAL QUESTIONS
Q II. What methods could be employed in the development of mashups aiming at
performance enhancement?
Q III. How models available in the literature can be used to assess performance
improvements obtained with mashups?

10. 10
Q I. What are the major causes for a poor performance in the execution of IT Service
Management processes?
Hypothesis: The employment of mashups enhances the performance of
human-centered ITSM processes
1INTRODUCTION
HYPOTHESIS AND FUNDAMENTAL QUESTIONS
Q II. What methods could be employed in the development of mashups aiming at
performance enhancement?
Q III. How models available in the literature can be used to assess performance
improvements obtained with mashups?

11. 11
Hypothesis: The employment of mashups enhances the performance of
human-centered ITSM processes
1INTRODUCTION
HYPOTHESIS AND FUNDAMENTAL QUESTIONS
Q I. What are the major causes for a poor performance in the execution of IT Service
Management processes?
Q II. What methods could be employed in the development of mashups aiming at
performance enhancement?
Q III. How models available in the literature can be used to assess performance
improvements obtained with mashups?

12. 12
Hypothesis: The employment of mashups enhances the performance of
human-centered ITSM processes
1INTRODUCTION
HYPOTHESIS AND FUNDAMENTAL QUESTIONS
Q III. How models available in the literature can be used to assess performance
improvements obtained with mashups?
Q II. What methods could be employed in the development of mashups aiming at
performance enhancement?
Q I. What are the major causes for a poor performance in the execution of IT Service
Management processes?

13. 13
BOUNDARIES OF THE INVESTIGATION

14. A service is a means of delivering value to customers
by facilitating outcomes customers want to achieve
without the ownership of specific costs and risks
Boundaries of the Investigation
ITSM Scenario
2
14
Supplier CustomerService
Definition

15. Boundaries of the Investigation
ITSM Scenario
2
15
Supplier CustomerService
Service
Strategy
Service
Design
Service
Transition
Service
Operation

16. Boundaries of the Investigation
Investigated ITSM Process
2
16
Supplier CustomerService
Service
Strategy
Service
Design
Service
Transition
Service
Operation
Functions
• IT Operations
Management
• Application Management
• Technical management
• Service Desk
Processes
• Event Management
• Access Management
• Problem Management
• Incident Management
• Request Fulfillment

17. Boundaries of the Investigation
Investigated ITSM Process
2
17
Supplier CustomerService
Service
Strategy
Service
Design
Service
Transition
Service
Operation
Functions
• IT Operations
Management
• Application Management
• Technical management
• Service Desk
Processes
• Event Management
• Access Management
• Problem Management
• Incident Management
• Request Fulfillment

18. Boundaries of the Investigation
Investigated ITSM Process
2
18
Supplier CustomerService
Service
Strategy
Service
Design
Service
Transition
Service
Operation
Request Fulfillment
Dispatcher System Administrators
Requests

19. Boundaries of the Investigation
Investigated ITSM Process
2
19
Supplier CustomerService
Service
Strategy
Service
Design
Service
Transition
Service
Operation
Request Fulfillment
Dispatcher System Administrators
Requests

20. • Human behavior and performance are difficult to
model, and consequently, to optimize
• Even if the nature of work is exactly the same, a
human may execute it in a different way each time
– Interrupted by external factors
– Usually employ different systems
– Execute multiple and complex tasks
– May be overloaded
Boundaries of the Investigation
Performance
2
20

21. • When the natural limits are reached or exceeded,
human operators can become:
– A bottleneck, slowing down the process execution
– Even more error-prone
Boundaries of the Investigation
Performance
2
21
Units of work performed per unit of time
Productivity
Non-defective units at the output of the process
Reliability

22. • When the natural limits are reached or exceeded,
human operators can become:
– A bottleneck, slowing down the process execution
– Even more error-prone
Boundaries of the Investigation
Performance
2
22
Units of work performed per unit of time
Productivity
Non-defective units at the output of the process
Reliability

23. • When the natural limits are reached or exceeded,
human operators can become:
– A bottleneck, slowing down the process execution
– Even more error-prone
Boundaries of the Investigation
Performance
2
23
Units of work performed per unit of time
Productivity
Non-defective units at the output of the process
Reliability

24. • The focus is on steps that can be measured through
instrumentation or observation, and can be
improved through redesign and partial automation
• The performance analysis covers the steps that a
human follows to execute ITSM processes
• The unpredictable nature of external events that
affect human behavior is not addressed
Boundaries of the Investigation
Performance
2
24

25. 25
PRODUCTIVITY

26. • Inefficiencies are portions of a service management
process characterized by suboptimal execution of
activities
• They can appear at different levels of analysis
– Lower level: inefficiencies due to the mechanical
execution involved in performing the activity
– Higher level: inefficiencies due to the complexity of the
activity itself
PRODUCTIVITY
INEFFICIENCIES
3

27. • We collected descriptions of the tasks performed by
a group of operators involved in a common activity
and analyzed both levels of inefficiencies
– Basic: context-switching, locating data, entering data
– Information Management: copy/paste, consistency
checks, information lookups
– Skill-dependent: retaining information, combining
information, data transformation
– Synchronization: contacting a person, becoming aware
PRODUCTIVITY
INEFFICIENCIES
3

28. • These inefficiencies can be tackled by the adoption
of mashups and, more specifically, by using
Mashup Patterns
• Mashup Patterns are general reusable solutions to
a commonly occurring problem, which:
– Provide tested and proven development improvements
– Allow mashups to be developed quickly
PRODUCTIVITY
MASHUPS
3

29. PRODUCTIVITY
MASHUPS
3
Housing Maps Chicago Crimes
Retrieve data
(houses)
Retrieve data
(crimes)
Display on
Google maps
Display on
Google maps

30. PRODUCTIVITY
MASHUPS
3• Candidate mashup patterns:
– Alerter: periodically monitors a system of interest on
behalf of the user and sends notifications
– Importer: abstracts the different methods used to access
the external data
– Transform: enables the processing of certain types of
data
– Displayer: presents information from multiple sources as
independent widgets

31. • The Keystroke-Level Model (KLM) is used to
measure lower level inefficiencies
– Predicts the time an expert user takes to perform a task
on a computer system
– It is based on the sequence of keystroke-level actions the
user performs
PRODUCTIVITY
QUANTITATIVE MODELING
3

32. • The Complexity Model is employed to account for
the higher level inefficiencies
– Evolved from a methodology for quantitative
benchmarking of configuration complexity to a model of
configuration activity based on goals, procedures, and
actions
– Provides a set of metrics, e.g., execution, parameter, and
memory
PRODUCTIVITY
QUANTITATIVE MODELING
3

33. 33
PRODUCTIVITY
QUANTITATIVE MODELING
3
Total time
Human-computer
interactions
Cognitive
actions
Analyst Domain
Expert

34. 34
RELIABILITY

35. • A planned sequence of physical or mental activities
that fails to obtain a result
• May be of two types:
– Involuntary actions (i.e., slips and lapses): are those that
deviate from planned intentions and, thus, do not reach
their goals
– Intentional actions (i.e., mistakes and violations): are
performed consciously but the desired result is not
achieved
35
RELIABILITY
HUMAN ERROR
4

36. • We identified a set of common error-prone activities
in ITSM:
– Action: are actions that change the state of the system
– Retrieval: are failures to retrieve correct information, to
be used in a further step
– Checking: occur when the operator fails to check some
information
– Decision: occur when the operator has to make an
explicit choice between multiple alternatives
– Communication: occur when the operator fails to pass
information to another person
36
RELIABILITY
HUMAN ERROR
4

37. • Mashup patterns, can be used to cope with human
errors by providing a set of proven and reusable
solutions
• We introduce a new type of mashup basic operator,
(i.e., Error Prevention modules):
– Buffer: allows developers to specify for how long time a
specific action should be delayed before being executed
– Forcing: allows developers to introduce confirmation
points in the process workflow
37
RELIABILITY
MASHUPS
4

38. • HEART is a technique used to quantify human
reliability of specific tasks
– It is considered the most comprehensive method in the
field of Human Reliability Assessment
– Specifies the factors that can affect human performance,
thus making it less reliable (e.g., distractions, overload)
– We employed Linguistic Variables to represent the
Assessed Proportion of Affect
• Event Tree Analysis is used to determine the
probability of failure in a sequence of events
38
RELIABILITY
QUANTITATIVE MODELING
4

39. 39
RELIABILITY
QUANTITATIVE MODELING
4
Analyst Domain
Expert
Task HEP
1 6.7
2 2.1
3 4.5
!"##$%%&
!"##$%%&
' "(#)*$&+&
' "(#)*$&,&
' "(#)*$&-&
' "(#)*$&.&/012&
/012&
/012&
/012&
!"##$%%&
!"##$%%& ' "(#)*$&3&
4##15 $6 (&
!#$6 071)%&
HEART

40. 40
EVALUATION

41. 41
EVALUATION
DISPATCH PROCESS
5
Service
Strategy
Service
Design
Service
Transition
Service
Operation
Request Fulfillment
Dispatcher System Administrators
Requests

42. 42
Problem
Ticket
Incident
Ticket
Change
Ticket
Low-level group
(Simple tickets)
High-level group
(Higher complexity
tickets)
Mid-level
group
(Root
Analysis,
complex
problems,
etc.)
Incoming
demand
Segmentation by
complexity
Work group structure based
On ticket complexity
Dispatcher
Other
Dispatcher
EVALUATION
DISPATCH PROCESS
5

43. 43
EVALUATION
DISPATCH PROCESS
5
1) Open Ticket
(ETS)
2) Analyses if the
Ticket was misrouted
(ETS)
4) Forwards to other
team (ETS)
5) Analyses the skill
level to solve the ticket
(ETS)
7) Request for more
resources
(e-mail)
8) Import the ticket
(ETS, ITS)
9) Searches for the SA
With the right skills and
Availability (ITS)
10) Talk with the SA
(in person)
11) Makes the
assignment
(ETS, ITS)
3) Is the
ticket
correct?
6) Have
enough
resources?
• Name
• Description
• Severity
• Workload
• Skills

44. 44
EVALUATION
MASHUP-BASED DISPATCH SYSTEM
5
1) Open Ticket
(ETS)
2) Analyses if the
Ticket was misrouted
(ETS)
4) Forwards to other
team (ETS)
5) Analyses the skill
level to solve the ticket
(ETS)
7) Request for more
resources
(e-mail)
8) Import the ticket
(ETS, ITS)
9) Searches for the SA
With the right skills and
Availability (ITS)
10) Talk with the SA
(in person)
11) Makes the
assignment
(ETS, ITS)
3) Is the
ticket
correct?
6) Have
enough
resources?
• Name
• Description
• Severity
• Workload
• Skills
Alerter Pattern
{New Ticket}
Displayer Pattern
{name, description}
Displayer Pattern
{severity}
Displayer Pattern
{workload, skills}
Importer
Pattern
Transformer
Pattern
Input operator
{System Admin}

45. 45
EVALUATION
MASHUP-BASED DISPATCH SYSTEM
5
1) Open Ticket
(ETS)
2) Analyses if the
Ticket was misrouted
(ETS)
4) Forwards to other
team (ETS)
5) Analyses the skill
level to solve the ticket
(ETS)
7) Request for more
resources
(e-mail)
8) Import the ticket
(ETS, ITS)
9) Searches for the SA
With the right skills and
Availability (ITS)
10) Talk with the SA
(in person)
11) Makes the
assignment
(ETS, ITS)
3) Is the
ticket
correct?
6) Have
enough
resources?
• Name
• Description
• Severity
• Workload
• Skills
Alerter Pattern
{New Ticket}
Displayer Pattern
{name, description}
Displayer Pattern
{severity}
Input operator
{System Admin}
Displayer Pattern
{workload, skills}
Importer
Pattern
Transformer
PatternBuffer
Operator
Forcing
Operator

46. 46
EVALUATION
MASHUP-BASED DISPATCH SYSTEM
5

47. • We performed a series of time measurements
among five dispatchers in a service delivery center
– Using a stopwatch, we took 10 time measurements for
each assignment process and its individual tasks
47
EVALUATION
PRODUCTIVITY ASSESSMENT
5
0
30
60
90
120
150
180
210
240
2 3 5 6 8 9 11
Timeavg.(seconds)
Task #

48. • The KLM model was determined through
observation of user interactions
48
5EVALUATION
PRODUCTIVITY ASSESSMENT
Copy and Paste Operation Time (sec)
Decide to do the task M 1.35 sec
Change Window (mouse) P + K 1.3 sec
Copy text P + B + P + B 2.4 sec
Paste text P + K + H + K + K 1.9
Ttask8 = M + Fields * CopyPaste
= M + Fields * (M + Change Window + CopyText + ChangeWindow + PasteText )
= 1.35 + 10 * (1.35 + 1.3 + 2.4 + 1.3 + 1.9)
= 83.85 sec

49. 49
5EVALUATION
PRODUCTIVITY ASSESSMENT
• The complexity time can be obtained by subtracting
the time predicted by the KLM model
0
20
40
60
80
100
120
140
160
1 2 3 5 6 8 9 11
Time(Seconds)
Task #
Complexity
KLM

50. • By interviewing dispatchers, it was possible to
determine values for all the memory and decision
metrics
50
EVALUATION
PRODUCTIVITY ASSESSMENT
5
0
2
4
6
8
10
12
1 2 3 5 6 8 9 11
Complexuty
Task #
Decision
Memory Size
Memory Latency
Memory Depth

51. 51
5EVALUATION
PRODUCTIVITY ASSESSMENT
• It’s required to evaluate how the model’s quality
changes as new metrics are included in the
evaluation
Step Added Metric R2 RMSE
1 Decision 0.01 39.09
2 Memory size 0.94 24.78
3 Memory latency 0.96 4
4 Memory depth 0.84 5.04

52. 52
5EVALUATION
PRODUCTIVITY ASSESSMENT
Step Added Metric R2 RMSE
1 Decision 0.01 39.09
2 Memory size 0.94 24.78
3 Memory latency 0.96 4
4 Memory depth 0.84 5.04
• It’s required to evaluate how the model’s quality
changes as new metrics are included in the
evaluation

53. 53
5EVALUATION
PRODUCTIVITY ASSESSMENT
• It was possible to predict the time (spend at each
task) associated with the complexity encountered in
carrying out the tasks of the assignment process
• The model can
explain 96% of
the time
variability
0
10
20
30
40
50
60
70
1 2 3 5 6 8 9 11
Time(seconds)
Task #
Complexity
Real

54. 54
• Once the time measures are estimated, it’s possible
to evaluate productivity enhancement by using the
mashup’s technology
5EVALUATION
PRODUCTIVITY ASSESSMENT
0
20
40
60
80
100
120
140
160
1 2 3 5 6 8 9 11
Time(Seconds)
Task #
Real measurements
(without mashups)
Predicted by model
(without mashups)
Predicted by model
(with mashhups)

55. 55
• It was possible to
identify the most
common failures in
dispatch
• The number of human
errors accounted is 10
for each 150 executions
(~6.6%)
5EVALUATION
RELIABILITY ASSESSMENT

56. 56
5EVALUATION
RELIABILITY ASSESSMENT
EPCs
HEART
effect
Assessed proportion
of effect
Assessed
effect
Channel overload x 6 Medium 3.51
Suppressing information x 9 Medium 5.02
No veracity checks x 2 Low 1.18
No means of conveying info x 8 Low 2.28
• Once the process workflow and root causes of
problems are determined, the next step is the
determination of Human Error Probability (HEP) of
each activity in the workflow
Task 5: Analyses the skill level to solve the ticket

57. 5EVALUATION
RELIABILITY ASSESSMENT
HEP[final] = HEP * (3.51 * 5.02 * 1.18 * 2.28)
= 0.0004 * 47.41
= 0.019 = 1.9%
Failure Task HEP
Accepts misrouted ticket 3 0.002391857
Underestimates skills 5 0.019094864
Imports wrong info 8 0.022549899
Fails to detect SA 9 0.019094864
Wrong assignment 11 0.002391857 P[failure] = 0.06401
= 6.4%
57

58. 58
• The new interaction elements and mashup patterns
are feasible mechanisms to reduce the occurrence
of human errors
5EVALUATION
RELIABILITY ASSESSMENT
EPCs
HEART
effect
Assessed proportion
of effect
Assessed
effect
Suppressing information x 9 Low 2.46
No veracity checks x 2 High 1.18
No means of conveying info x 8 Low 2.28
Task 5: Analyses the skill level to solve the ticket (after mashup usage)

59. 59
5EVALUATION
RELIABILITY ASSESSMENT
Failure Task HEP
Accepts misrouted ticket 3 0.00091324
Underestimates skills 5 0.00412552
Imports wrong info 8 0.00000000
Fails to detect SA 9 0.00091324
Wrong assignment 11 0.00476030
HEPfinal = HEP * (2.46 * 1.83 * 2.28)
= 0.0004 * 10.31
= 0.0041 = 0.41%
Pfailure = 0.011004691
= 1.1 %

60. 60
• The results show the reduction from 6.4% to 1.1% in
the probability of human error
5EVALUATION
RELIABILITY ASSESSMENT
0
0.005
0.01
0.015
0.02
0.025
1 2 3 4 5
HEPvalues
Failure #
Without mashups With mashups

61. • The Pearson product-moment correlation r indicate
a fair and positive relationship among the variables
61
5EVALUATION
CORRELATION ANALYSIS
0.00%
1.00%
2.00%
3.00%
4.00%
5.00%
6.00%
7.00%
0 50 100 150 200 250 300
CumulativeErrorProbability
(percentage)
Execution Time (seconds)
Mashups
Without Mashups

62. 62
CONCLUSIONS AND FUTURE WORK

63. • Improvements in productivity and reliability provided
by the application of mashups demonstrate its
viability as a means for improving IT Service
Management
• The combined model allowed us to tackle lower
level inefficiencies of human-computer interactions,
and higher level inefficiencies of performing IT tasks
and subtasks
63
6CONCLUSIONS AND FUTURE WORK
FINAL CONSIDERATIONS

64. • The good fit between the real measurements with
the times predicted by the combined model
indicates its feasibility in predicting labor costs
savings
• The presented patterns and interaction elements
avoided the occurrence of action and retrieval
errors
64
6CONCLUSIONS AND FUTURE WORK
FINAL CONSIDERATIONS

65. • Inefficiencies due to the complexity and mechanical execution involved in
performing the activity.
• Four groups of inefficiencies: basic, information management, skill-
dependent, and synchronization.
• Errors can occur due to: attention span, memory, situation awareness, and
personal resources.
• Five common error-prone activities: action, retrieval, checking, decision, and
communication errors.
65
6CONCLUSIONS AND FUTURE WORK
FINAL CONSIDERATIONS
Q I. What are the major causes for a poor performance in the execution of IT Service
Management processes?
Hypothesis: The employment of mashups enhances the performance of
human-centered ITSM processes

66. • KLM provides a wealth of detail at the lower level human-computer interactions,
while the Complexity Model addresses both levels of inefficiencies.
• HEART can be used to quantify the human reliability of individual tasks. ETA was
used to evaluate the overall human error probability of ITSM processes.
66
6CONCLUSIONS AND FUTURE WORK
FINAL CONSIDERATIONS
Q II. How models available in the literature can be used to assess performance
improvements obtained with mashups?
Hypothesis: The employment of mashups enhances the performance of
human-centered ITSM processes

67. • Mashup patterns to tackle inefficiencies and error-prone activities in service
management processes: Alerter, Importer, Transform, and Displayer (SANTOS et
al., 2011).
• A new type of interaction elements, called Error Prevention modules (SANTOS et
al., 2013).
67
6CONCLUSIONS AND FUTURE WORK
FINAL CONSIDERATIONS
Q III. What methods could be employed in the development of mashups aiming at
performance enhancement?
Hypothesis: The employment of mashups enhances the performance of
human-centered ITSM processes

68. • Validation of mashup patterns through additional
supporting cases
• Investigate financial models to assess loss due to broken
SLAs
• Automation of mashup’s development
• Inclusion of confidentiality mechanisms in the mashup’s
development
• Investigation of rollback and exception handling in the
mashup composition logic
68
6CONCLUSIONS AND FUTURE WORK
NEXT STEPS

69. • 2009:
– Rafael Santos Bezerra, Carlos Raniery Paula dos ; Leandro Márcio
Bertholdo ; Lisandro Zambenedetti Granville ; Liane Margarida
Rockenbach Tarouco . Um Sistema de Gerenciamento de Redes
Baseado em Mashups. SBRC 2009
– Rafael Santos Bezerra, Carlos Raniery Paula dos ; Leandro Márcio
Bertholdo ; Lisandro Zambenedetti Granville ; Liane Margarida
Rockenbach Tarouco . Um Sistema de Gerenciamento de Redes
Baseado em Mashups. Revista Brasileira de Redes de Computadores
e Sistemas Distribuídos (RESD), v. 2, 2009.
69
6CONCLUSIONS AND FUTURE WORK
PUBLICATIONS

70. • 2010:
– Rafael Santos Bezerra, Carlos Raniery Paula dos ; Lisandro
Zambenedetti Granville ; Liane Margarida Rockenbach Tarouco . On
the Feasibility of Web 2.0 Technologies for Network Management:
A Mashup-Based Approach. 12th Network Operations & Management
Symposium (NOMS), 2010, Osaka, Japan
– Carlos Raniery Paula dos Santos, Rafael Santos Bezerra, João
Marcelo Ceron, Lisandro Zambenedetti Granville, Liane Margarida
Rockenbach Tarouco. On Using Mashups for Composing Network
Management Applications. IEEE Communications Magazine, Vol. 48,
Issue 12, December 2010
– Carlos Raniery Paula dos Santos, Rafael Santos Bezerra, João
Marcelo Ceron, Lisandro Zambenedetti Granville, Liane Margarida
Rockenbach Tarouco. Botnet Master Detection Using a Mashup-
based Approach. 6th IEEE International Conference on Network and
Service Management (CNSM), 2010, Niagara Falls, Canada
70
6CONCLUSIONS AND FUTURE WORK
PUBLICATIONS

71. • 2011:
– Carlos Raniery Paula dos Santos, Rafael Santos Bezerra, João
Marcelo Ceron, Lisandro Zambenedetti Granville, Liane Margarida
Rockenbach Tarouco. Identifying Botnet Communications Using a
Mashup-based Approach. LANOMS, 2011, Quito, Equador
– Carlos Raniery Paula dos Santos, Winnie Cheng, Rafael Santos
Bezerra, Lisandro Zambenedetti Granville, Nikos Anerousis. A Data
Confidentiality Architecture for Developing Management Mashups.
12th IFIP/IEEE International Symposium on Integrated Network
Management, 2011, Dublin, Ireland
– Carlos Raniery P. dos Santos, Lisandro Zambenedetti Granville, Winnie
Cheng, David Loewenstern, Larisa Shwartz, Nikos Anerousis.
Performance Management and Quantitative Modeling of IT Service
Processes Using Mashup Patterns. 7th International Conference on
Network and Services Management (CNSM 2011), 2011, Paris, France
71
6CONCLUSIONS AND FUTURE WORK
PUBLICATIONS

72. • 2012:
– Carlos Raniery Paula Dos Santos, Lisandro Zambenedetti Granville,
Nikolaos Anerousis, David Matthew Loewenstern, Louis Jonh Percello,
Winnie Cheng, Larisa Shwartz. Performance Management and
Quantitative Modeling of IT Service Processes Using Mashup
Patterns. US Patent Pending
72
6CONCLUSIONS AND FUTURE WORK
PUBLICATIONS

73. • 2013:
– Carlos Raniery Paula dos Santos, Lisandro Zambenedetti Granville,
Larisa Shwartz, Nikos Anerousis, David Loewenstern. Quality
Improvement and Quantitative Modeling – Using Mashups for
Human Error Prevention. 13th IFIP/IEEE Symposium on Integrated
Network and Service Management (IM 2013), 2013, Ghent, Belgium
73
6CONCLUSIONS AND FUTURE WORK
PUBLICATIONS

74. • Virtual Nodes Monitoring based on Mashup
– This paper provides a mashup-based mechanism
to monitor virtualized networks.
74
6CONCLUSIONS AND FUTURE WORK
NEXT PUBLICATIONS

75. • A Mashup-based Solution for Botnet
Mitigation
– This paper proposes a modular architecture
based on the dynamic integration of pre-existing
tools to achieve a more efficiently detection
solution.
75
6CONCLUSIONS AND FUTURE WORK
NEXT PUBLICATIONS

76. • Survey
– This work aims at gathering and organizing the
knowledge about mashups properties applied to
multiple management scenarios
76
6CONCLUSIONS AND FUTURE WORK
NEXT PUBLICATIONS

77. • Automation of mashups’ development
– Considering that the ITSM operators have a
strong knowledge of the process they use to
perform, but may not have expertise in mashups
development, the use of an automated
development system may ultimately provide a
significantly improved orchestration of the
process
77
6CONCLUSIONS AND FUTURE WORK
NEXT PUBLICATIONS

78. • Overview of the Thesis
– This paper will present all the performance
metrics analyzed and show the key concepts of
this thesis
78
6CONCLUSIONS AND FUTURE WORK
NEXT PUBLICATIONS

79. Carlos Raniery Paula dos Santos
Orientador: Lisandro Zambenedetti Granville
Instituto de Informática – UFRGS
Inf.ufrgs.br/~crpsantos
Some questions….
79
Computer
Networks
Group
Thank you for your attention!

80. • [01] ISACA. Control Objectives for Information and related
Technologies (COBIT). 2008
• [02] OGC. Information Technology Infrastructure Library v3 (ITIL
v3). 2008
• [03] PULTORAK, D.; HENRY, C.; LEENARDS, P. MOF 4. 0: a
pocket guide: (microsoft operations framework). [S.l.]: Van
Haren Publishing, 2008. (Van Haren Series)
• [04] MILLIKEN, K. R.; CRUISE, A. V.; ENNIS, R. L.; FINKEL, A.
J.; HELLERSTEIN, J. L.; LOEB, D. J.; KLEIN, D. A.; MASULLO,
M. J.; VANWOERKOM, H. M.;WAITE, N. B. YES/MVS and the
automation of operations for large computer complexes. IBM
System Journal, [S.l.], v.25, p.159–180, June 1986.
• [05] CANDEA, G.; KICIMAN, E.; KAWAMOTO, S.; FOX, A.
Autonomous recovery in componentized Internet applications.
Cluster Computing, Hingham, MA, USA, v.9, p.175–190, April
2006.
80
References

81. • [06] KELLER, A.; HELLERSTEIN, J. L.; WOLF, J. L.; WU, K.
L.; KRISHNAN, V. The CHAMPS system: change
management with planning and scheduling. In: NETWORK
OPERATIONS AND MANAGEMENT SYMPOSIUM, 2004.
NOMS 2004. IEEE/IFIP, 2004. Anais. . . [S.l.: s.n.], 2004. v.1,
p.395–408 Vol.1.
• [07] LIKER, J. The Toyota way: 14 management principles
from the world’s greatest manufacturer. [S.l.]: McGraw-Hill,
2004.
• [08] OGC. Information Technology Infrastructure Library v3
(ITIL v3). 2008.
• [09] Michael Ogrinz. Mashup Patterns: Designs and
Examples for the Modern Enterprise. 2009. ISBN 978-
0321579478
81
References

82. • [10] CARD, S. K.; NEWELL, A.; MORAN, T. P. The
Psychology of Human-Computer Interaction. Hillsdale, NJ,
USA: L. Erlbaum Associates Inc.,2000
• [12] BROWN, A. B.; HELLERSTEIN, J. L. An approach to
benchmarking configuration complexity. In: IN
PROCEEDINGS OF THE 11TH ACM SIGOPS EUROPEAN
WORKSHOP, 2004. Anais. . . [S.l.: s.n.], 2004.
• [13] DIAO, Y.; KELLER, A. Quantifying the Complexity of IT
Service Management Processes. In: DSOM, 2006. Anais. . .
[S.l.: s.n.], 2006. p.61–73
• [14] DIAO, Y.; KELLER, A.; PAREKH, S. S.; MARINOV, V. V.
Predicting Labor Cost through IT Management Complexity
Metrics. In: INTEGRATED NETWORK MANAGEMENT, 2007.
Anais. . . [S.l.: s.n.], 2007. p.274–283
82
References

83. • [15] REASON, J. Human Error. [S.l.]: Cambridge [England] ;
New York : Cambridge University Press, 1990. xv, 302 p.,
1990.
• [16] KIRWAN, B. The validation of three human reliability
quantification techniques THERP, HEART and JHEDI: part 1 -
ttechnique descriptions and validation issues. Applied
Ergonomics, [S.l.], v.27, n.6, p.359 – 373, 1996
83
References

84. 84
Browser
Browser
asdasdasdasdasdasda
sdasdasdasdasdasdas
dasdasdasdasdasdads
asdasdadsasda
asdasdasdasdasdasda
sdasdasdasdasdasdas
dasdasdasdasdasdads
asdasdadsasda
Mashup
System
Developer
End User Content Providers
1INTRODUCTION
MASHUPS

85. 85
RELIABILITY
MASHUPS
4 Adaptation Adaptation
Operation
(merge)
Forcing
Buffer
Adaptation
External
System
Mashup System

86. 86
PRELIMINARY EVALUATION
DISPATCH PROCESS
5

87. • The focus is on the Request Fulfillment process
– Responsible for carrying out service requests, and that
interfaces with Service Desk, Incident Management, and
Change Management
• This process includes two types of human
operators:
– System administrators: technical personnel with
knowledge to resolve specific requests
– Dispatchers: responsible for monitoring new requests,
dispatching the requests to the appropriate SA, and
monitoring compliance with SLAs
87
PRELIMINARY EVALUATION
DISPATCH PROCESS
5

88. • Interaction Components:
– Visual: represent way data can be displayed,
such as map, tables, and trees
– Control: represent basic programming logic, such
as loops, and conditions
– Operation: perform operations over retrieved
information, such as filtering, merging, and
arithmetic operations
– Adaptation: represent external resources, these
are created based on wrapper meta-information
stored
– Reuse: this type of component represent existing
mashups, enabling their reuse in other masups
88
Mashup System

89. • Effectiveness: how to match the customers
requirements and what the service provides in fact
• Cost: may be defined in SLAs
• Security: probability of information leakage
89
Other metrics

90. • Checklists are increasingly being used in Scottish
hospitals, for example in pre-operative settings
• Minimize staff interruptions and distractions
• Prevent errors: procedures, training, UI design
(allow only valid choices)
• Recover errors: undo capability, confirmation
90
Other system improvements

91. 91
Mashup Patterns

92. 92
Gesture Time
K Keying 0.2 sec
B Holding/Releasing key 0.1 sec
P Pointing 1.1 sec
H Homing 0.4 sec
M Mentally Preparing 1.35 sec
KLM

93. • KLM model: provides a wealth of detail at the lower level of
human-computer interactions
• Complexity model: besides it addresses both levels of
inefficiencies, we discard all the complexity metrics except
the memory and decision metrics, which capture higher level
potential inefficiencies not addressed by KLM
93
Total time spend by a human
Human-computer
interactions
Cognitive
actions
Quantitative Modeling

94. • The approach can be summarized in three steps:
– Assessing the complexity and timing a baseline scenario
– Construction of the regression model and its quality
evaluation
– Employing the model to predict labor costs
94
Complexity Model