System of Systems Engineering (SoSE), System “ilities” (Reliability, Availability, Maintainability, and Changeability), State Series, System Evolution Analytics, System Network Evolution Rules, System Network Complexity, System Evolution Recommender Service Evolution Analytics

1. Advance Systems Engineering Topics
by
Dr.Animesh Chaturvedi
Assistant Professor: LNMIIT Jaipur
Post Doctorate: King’s College London &TheAlanTuring Institute
PhD: IIT Indore

2. ❑ System of Systems Engineering (SoSE),
❑ System “ilities” (Reliability,Availability, Maintainability, and Changeability),
❑ State Series,
❑ System EvolutionAnalytics,
❑ System Network Evolution Rules,
❑ System Network Complexity,
❑ System Evolution Recommender
❑ Service EvolutionAnalytics
Advance Systems Engineering Topics

3. System of Systems Engineering (SoSE)

4. System of Systems Engineering (SoSE)
 Maier (1998) key characteristics of SoS:
 operational independence, managerial independence,
 geographical distribution, emergent behavior, and
 evolutionary development processes
https://www.sebokwiki.org/wiki/Systems_of_Systems_(SoS)
Systems tend to ... Systems of systems tend to ...
Clear set of stakeholders Multiple levels of stakeholders with mixed and possibly
competing interests
Clear objectives and purpose Multiple, and possibly contradictory, objectives and purpose
Clear operational priorities, with escalation to
resolve priorities
Multiple, and sometimes different, operational priorities with no
clear escalation routes
A single lifecycle Multiple lifecycles with elements being implemented
asynchronously
Clear ownership with the ability to move resources
between elements
Multiple owners making independent resourcing decisions

5. Systems of Systems (SoS)
ISO/IEC/IEEE 21839 (ISO, 2019)
 System of Systems (SoS) — Set of systems or system elements that interact to
provide a unique capability that none of the constituent systems can accomplish on
its own. Note: Systems elements can be necessary to facilitate the interaction of the
constituent systems in the system of systems
 Constituent Systems — Constituent systems can be part of one or more SoS.
Note: Each constituent is a useful system by itself, having its own development,
management goals and resources, but interacts within the SoS to provide the
unique capability of the SoS.
https://www.sebokwiki.org/wiki/Systems_of_Systems_(SoS)

6. System of Systems Engineering (SoSE)
 Systems of systems are systems where two or more of the constituent systems are
independently managed and governed.
 Reductionism as an engineering method breaks down because of the inherent
complexity of systems of systems.
 Reductionism assumes clear system boundaries, rational decision making and well-
defined problems. None of these are true for systems of systems.
 Governance and management policies must be designed in parallel.
Sommerville, Ian. "Software engineering 10th Edition." (2015).

7. System of Systems Engineering (SoSE)
 Three types of complexity in systems of systems –
 technical complexity,
 managerial complexity and
 governance complexity.
 The key stages of the SoS development process are
 conceptual design,
 system selection,
 architectural design,
 interface development, and
 integration and deployment.
Sommerville, Ian. "Software engineering 10th Edition." (2015).

8. SoSE development processes
Sommerville, Ian. "Software engineering 10th Edition." (2015).

9. SoSE development processes
 Conceptual design is the activity of creating a high-level vision for a system, defining
essential requirements and identifying constraints on the overall system.
 System selection,where a set of systems for inclusion in the SoS is chosen.
 Political imperatives and issues of system governance and management are often the key
factors that influence what systems are included in a SoS.
 Architectural design where an overall architecture for the SoS is developed.
 Interface development – the development of system interfaces so that the constituent
systems can interoperate.
 Integration and deployment – making the different systems involved in the SoS work
together and interoperate through the developed interfaces.
Sommerville, Ian. "Software engineering 10th Edition." (2015).

10. SoSE development processes
 System deployment means putting the system into place in the organizations
concerned and making it operational.
Sommerville, Ian. "Software engineering 10th Edition." (2015).

11. System “ilities”
(Reliability, Availability, Maintainability, & Changeability)

12. Emergent properties
 The emergent properties of a system are characteristics of the system as a whole
rather than of its component parts.
 They include properties such as
 performance,
 reliability,
 usability,
 safety and
 security.
Sommerville, Ian. "Software engineering 10th Edition." (2015).

13. System “ilities”
Accessibility
Accountability
Adaptability
Administrability
Affordability
Auditability
Availability
Credibility
Compatibility
Configurability
Correctness
Customizability
Installability
Integrity
Interchangeability
Interoperability
Learnability
Maintainability
Manageability
Mobility
Modifiability
Modularity
Operability
Portability
Predictability
Provability
Supportability
Securability
Simplicity
Stability
Survivability
Sustainability
Tailorability
Testability
Timeliness
Traceability
Ubiquity
Understandability
Upgradability
Usability
Versatility
Recoverability
Reliability
Repeatability
Reproducibility
Resilience
Responsiveness
Reusability
Robustness
Safety
Scalability
Sustainability
Serviceability
Debugability
Degradability
Determinability
Demonstrability
Dependability
Deployability
Distributability
Durability
Effectiveness
Evolvability
Extensibility
Fidelity
Flexibility

14. Examples of emergent properties
Property Description
Reliability System reliability depends on component reliability but unexpected interactions can
cause new types of failures and therefore affect the reliability of the system.
Repairability This property reflects how easy it is to fix a problem with the system once it has been
discovered. It depends on being able to diagnose the problem, access the components
that are faulty, and modify or replace these components.
Security The security of the system (its ability to resist attack) is a complex property that cannot
be easily measured. Attacks may be devised that were not anticipated by the system
designers and so may defeat built-in safeguards.
Usability This property reflects how easy it is to use the system. It depends on the technical
system components, its operators, and its operating environment.
Volume The volume of a system (the total space occupied) varies depending on how the
component assemblies are arranged and connected.
Sommerville, Ian. "Software engineering 10th Edition." (2015).

15. Reliability, Availability, & Maintainability (RAM)
 RAM are system attributes that affect system life-cycle costs
 RAM are interest to engineers, logisticians, and users.
 Systems Engineering, Product Management (Life Cycle Cost andWarranty), Quality Assurance
 Reliability: measures probability of a system or its component performing their
expected function for the requirements without failure for a defined period of time.
 Maintainability: measures probability of a system or its component can be repaired in a
defined environment within a specified period of time. Increased maintainability implies
shorter repair times
 Availability: the probability of a repairable system or its component is operational at a
given point in time under a given set of environmental conditions.Availability depends on
reliability and maintainability
https://www.sebokwiki.org/wiki/Reliability,_Availability,_and_Maintainability

16. Reliability
 Collectively, they affect economic life-cycle costs of a system and its utility.
 Description of the function, the environment, the time scale, and what constitutes
a failure.
 Emergent property: because of component inter-dependencies, faults can be
propagated through the system.
 System failures often occur because of unforeseen inter-relationships between
components.
 It is practically impossible to anticipate all possible component relationships.
https://www.sebokwiki.org/wiki/Reliability,_Availability,_and_Maintainability

17. Availability
 Availability can be calculated from the total operating time and the downtime, or
in the alternative, as a function of MTBF and MTTR (MeanTimeTo Repair.)
 Top,Tot is the total operating time
 Tdown,tot is the total down time
 MeanTimeTo Failure (MTTF) for non-restorable systems, or MeanTime Between
Failures (MTBF for restorable systems are used)
 MTTR (MeanTimeTo Repair.)
https://www.sebokwiki.org/wiki/Reliability,_Availability,_and_Maintainability

18. Maintainability
 correct defects or their cause, repair or replace faulty components,
 prevent unexpected working conditions, maximize a product's useful life,
 maximize efficiency, reliability, and safety, meet new requirements,
 make future maintenance easier, or cope with a changing environment.
 involves continuous improvement of a system
 Maintainability is often characterized in terms of the exponential distribution and
the mean time to repair and be similarly calculated, i.e.,
 whereTdown,Tot is the total down time and noutages is the number of outages.
https://en.wikipedia.org/wiki/Maintainability
https://www.sebokwiki.org/wiki/Reliability,_Availability,_and_Maintainability

19. Changeability
 System “changeability” (Ross et al.) define it as system ilities
 Change Agent: Instigator, or force, which employs a given change mechanism in
order to achieve a desired change effect
 Change Effect: The difference in system states (performance or value) before
and after a change has taken place
 Change Objective: The specific approach / plan / goal / strategies employed to
achieve a desired change effect
 Change Enablers: (e.g. design elements) enable desired objective
 Change Considerations: Design considerations (e.g. conditions, resources,
constraints, etc.) applied to design / operational approaches
Ross, A.M., Rhodes, D.H., and Hastings, D.E., "Defining Changeability: Reconciling Flexibility, Adaptability, Scalability, Modifiability, and
Robustness for Maintaining Lifecycle Value," Systems Engineering, Vol. 11, No. 3, pp. 246-262, Fall 2008.

20. State Series

21. Defining Evolving System
We can define (Si, ERi, ti) for an evolving system represented as a State Series (SS), a collection of
states (or data points), such that SS = {S1, S2 … SN} at various time points {t1, t2, t3 … tN}. Each
state is represented as a series of evolution representor ER = {ER1, ER2… ERN}.
Evolving systems: “Solar Power Satellite”, “Evolving Software System”
Evolving state series: “software version series”, “natural-language document versions”
21
➢ Abstract of PhD Thesis on 7th Heidelberg Laurate Forum 2019 Proceedings (Informal Publication). The proceeding also
contains work details of the 23 Laurates i.e. Turing award winners, Field Medalist, and Abel Prize winners. The HLF is an
event conducted by ACM, IMU, DNVA etc.
➢ Abstract of PhD Thesis on Dagstuhl Seminar 19401 Report (Informal Publication).
➢ Animesh Chaturvedi, and Aruna Tiwari. "System Network Complexity: Network Evolution Subgraphs of System State Series."
IEEE Transactions on Emerging Topics in Computational Intelligence (2018). DOI: 10.1109/TETCI.2018.2848293

22. Evolving System Examples
22

23. 23
https://www.filmsite.org/200films.html

24. Evolving Web Service System Examples
24

25. System Evolution Analytics

26. Problem definition
It is challenging to study the changeability and evolution in an evolving system. There are following
two challenges in evolving system mining and learning:
A. Generally, evolving systems are complex in nature, thus pre-processing of an evolving system is a
challenging task and requires a system domain expert.
B. The data mining or machine learning on single state of an evolving system is a straightforward
technique. However, it is challenging to process multiple states of an evolving system with the same
technique.
➢ Abstract of PhD Thesis on 7th Heidelberg Laurate Forum 2019 Proceedings (Informal Publication). The proceeding also
contains work details of the 23 Laurates i.e. Turing award winners, Field Medalist, and Abel Prize winners. The HLF is
an event conducted by ACM, IMU, DNVA etc.
➢ Abstract of PhD Thesis on Dagstuhl Seminar 19401 Report (Informal Publication).
26

27. System Evolution Analytics
27

28. Experiments on Evolving Systems
Domains of Evolving
System Evolving Systems “Source” and “Target” Entities Type of Connections Type of network
Applications of System
Evolution Analytics
(A)
Evolving Software System Hadoop HDFS-Core1 “Caller” and “Callee”
Procedures Procedural calls Call graph Software Evolution Analytics
(B)
Evolving
Natural-language systems
Bible Translation2
Words in “Source biblical
language” and “English variant”
languages
Translations Words Network
Natural-language Evolution
Analytics
Multi-sport Events3 Words in “Titles” (name) and
“Scopes” (region) of events Regional names Words network
(C)
Evolving Retail Market
System
Frequent Market
Basket4
Words in “Product description”
and Words in “Product
description”
Purchases Purchase network Market Evolution Analytics
(D)
Evolving IMDb movie
genre systems5
Positive sentiment6 of
movie genres5
“Positive words in names” and
“genres” of movies Sentiments Positive sentiment
network
Movie Evolution Analytics
Negative sentiment6 of
movie genres5
“Negative words in names” and
“genres” of movies Sentiments Negative sentiment
network
1. https://mvnrepository.com/artifact/org.apache.hadoop/hadoop-hdfs 2016.
2. https://en.wikipedia.org/wiki/List_of_English_Bible_translations Oct 2016.
3. https://en.wikipedia.org/wiki/List_of_multi-sport_events Oct 2016.
4. https://archive.ics.uci.edu/ml/datasets/Online+Retail Oct 2016.
5. http://www.imdb.com/interfaces/ Oct 2016.
6. https://www.cs.uic.edu/~liub/FBS/sentiment-analysis.html June 2017
28

29. System Network Evolution Rules

30. Stable Network Evolution Rule Mining for System
Stability and Changeability analysis
- Animesh Chaturvedi, Aruna Tiwari, and Nicolas Spyratos. “minStab: Stable Network Evolution Rule
Mining for System Changeability Analysis”. IEEE Transactions on Emerging Topics in Computation
Intelligence (2019). IEEE Computational Intelligence Society DOI:10.1109/TETCI.2019.2892734
- Animesh Chaturvedi, and Aruna Tiwari. “System Evolution Analytics: Evolution and Change Pattern
Mining of Inter-Connected Entities”. 48th IEEE International Conference on Systems, Man, and Cybernetics
(SMC), Miyazaki Japan, October 2018, pp. 3877-3882. IEEE SMC Society DOI:10.1109/SMC.2018.00750
30
• Evolution rule, Stable rule, Network Evolution Rule, and Stable Network Evolution Rule
• System Stability and Changeability Metrics
• Stable Network Evolution Rule Mining and Changeability Metric
• System Network Stability
• Experiments on Evolving Systems

31. Sequential rule mining
Sequential Rule Mining is a special kind of Association Rule Mining
A sequential rule indicates that if some event(s) occur, some other event(s) are likely to follow
with a given confidence or probability.
X ➔ Y if X appears, Y is likely to appear with a given confidence afterward
Each rule is found based on its support and its confidence.
An example: {Vivaldi}, {Mozart}, {Handel} ➔ {Berlioz}
Customer(s) who bought the music of Vivaldi, Mozart and Handel in that order, have then
bought the music of Berlioz
31
Fournier-Viger, P., Wu, C., Tseng, V., Cao, L., & Nkambou, R. “Mining Partially-Ordered Sequential Rules Common to Multiple Sequences”
IEEE Transactions on Knowledge and Data Engineering, vol. 27, no. 8, August 2015.

32. Sequential rule mining
1: {Vivaldi}, {Mozart}, {Handel}, {Berlioz}
2: {Mozart}, {Bach},{Paganini}, {Vivaldi}, {Handel}, {Berlioz}
3: {Handel}, {Vivaldi}, {Mozart}, {Ravel}, {Berlioz}
4: {Vivaldi}, {Mozart}, {Handel}, {Bach}, {Berlioz}
5: {Mozart}, {Bach}, {Vivaldi}, {Handel}
6: {Vivaldi}, {Handel}, {Mozart}, {Bach}
Consider {Vivaldi}, {Handel}, {Berlioz} Support 50%: 1, 2 and 4 out of six sequences
{Vivaldi}, {Mozart}, {Handel} ➔ {Berlioz}
Support 33%: two sequences (1 and 4) out of six sequences.
Confidence 100%: {Vivaldi}, {Mozart}, {Handel} appears in that order, it is followed by {Berlioz}
32
Fournier-Viger, P., Wu, C., Tseng, V., Cao, L., & Nkambou, R. “Mining Partially-Ordered Sequential Rules Common to Multiple Sequences”
IEEE Transactions on Knowledge and Data Engineering, vol. 27, no. 8, August 2015.

33. Proposed Definitions
33
A Network Rule (NR) in state Si is an expression of the form X→Y, where X is a subset of source
entities sSi and Y is a subset of target entities tSi.
A Network Evolution Rule (NER) in SS is an expression of the form X→Y, where X is a subset of
source entities sSS and Y is a subset of target entities tSS. Each NER has a characteristic stability
because it is distinct and interesting in one or more states.
◦ The stability count of X→Y in SS is denoted as stabCount(X→Y, SS), and defined by stabCount(X→Y, SS) =
n, where n is the number of states in SS that has X→Y as interesting NER.
◦ The stability of X→Y in SS is denoted as stab(X→Y, SS), and defined by stab(X→Y, SS) = stabCount(X→Y,
SS) ÷ card(SS), where card(SS) is the cardinality of SS i.e. number of states N in SS.
◦ The NER X→Y is defined to be stable in SS if its stability (or stability count) is greater minStab (or
minStabCount); such NERs are Stable Network Evolution Rules (SNERs).

34. 34

35. Stable Network Evolution Rule Mining and Changeability Metric:
The process-artifacts involved to intelligently compute the SNERs and the Changeability Metric.
Proposed Approach
35

36. The algorithm intelligently compute
• Stable Network Evolution Rules (SNERs) and
• Changeability Metric (CM)
Proposed Approach
36

37. Experiments on Evolving Systems
37

38. Experiments on Evolving Systems
38

39. Experiments on Evolving Systems
39
Software Evolution Analytics on Evolving Software Systems:
➢ Hadoop – HDFS available on Software Repository

40. Experiments on Evolving Systems
40
Natural-Language Evolution Analytics on two Evolving Natural-Language systems:
➢ List of Bible Translation system available on Wikipedia
➢ List of Multi-sport events system available on Wikipedia

41. Experiments on Evolving Systems
41
Market Evolution Analytics on Evolving Retail Market system:
➢ Frequent Market Basket system made from retail-market baskets available on UCI Repository

42. Experiments on Evolving Systems
42
Movie Evolution Analytics on two Evolving IMDb movie genre systems:
➢ Positive sentiment system made from IMDB movie-genre available on IMDb Repository
➢ Negative sentiment system made from IMDB movie-genre available on IMDb Repository

43. Conclusions
This introduced stability as a new measure for characterizing Network Evolution Rules (NERs) over
time.
Define the stability of an NER to be the percentage of states in SS in which the rule is interesting.
NER stable in SS if its stability exceeds a given threshold named as minimum stability (minStab).
An algorithm SNER Mining and Changeability Metric (SNERM_CM), which presented an approach to
retrieve NERs and Stable NERs (SNERs) in a set of states SS = {S1 , S2 ... SN } of an evolving system.
The purpose of SNERs is to study and compute the changeability property of an evolving system.
The SNERM_CM algorithm, we developed an intelligent tool,
Summarized experimental results to show various SNERs for some chosen value of minimum support
count (minSupCount), minimum confidence (minConf), and minimum stability count (minStabCount).
43

44. System Network Complexity

45. Network Evolution Subgraph (Graphlet or Motif)
Mining for System Stability, Changeability, and
Complexity analysis
- Animesh Chaturvedi, and Aruna Tiwari. "System Network Complexity: Network Evolution Subgraphs of
System State Series." IEEE Transactions on Emerging Topics in Computational Intelligence (2018).
DOI: 10.1109/TETCI.2018.2848293
- Animesh Chaturvedi, and Aruna Tiwari. “System Evolution Analytics: Evolution and Change Pattern
Mining of Inter-Connected Entities”. IEEE International Conference on Systems, Man, and Cybernetics
(SMC), Miyazaki Japan, October 2018, pp. 3877-3882. IEEE SMC Society DOI: 10.1109/SMC.2018.00750
45
• Network Evolution Subgraph (Graphlets and Motifs)
• System Stability and Changeability Metrics
• System State Complexity (SSC) & Evolving System Complexity (ESC)
• Proposed Framework
• Mining NEGs, NEMs, Changeability, and Stability
• System Network Complexity:
• Experiments on Evolving Systems

46. Subgraph Mining
Subgraph mining using a network of State Si. Subgraph can be Network Motifs or Graphlets
We call the retrieved doubleton set as subgraph information and denoted it as graphlet_info for
total M number of subgraphs
graphlet_info = j=0ΠM(Gj, freqj)
(G0, 25), (G2, 25), (G71, 50)
46
Example of subgraphs (graphlets) of size 4 (number of
nodes) as a set of subgraph key {G0, G1, G2, G3, ... G198}

47. Proposed Definitions
A network evolution subgraph is a subgraph that is common in the state series SS, such that it appears in
multiple states of the system with changing frequencies. Depending upon the properties, the network
evolution subgraph can be of three types:
- network evolution frequent subgraph: if subgraph is frequently occurring (with minimum support and confidence);
- network evolution motif: if subgraph is statistically recurrent in a network; and
- network evolution graphlet: if subgraph is induced subgraph in a network.
Let Gj is a network evolution graphlet (NEG). The Aggregate_freqj denotes aggregate frequency of a NEG
Gj. Aggregation could be of many types like mean, median, mode, sum, count, max, and min. The
arithmetic mean of frequency over N states for a NEG Gj is given by
Aggregate_freqj =
σ𝑖=1
N
𝑓𝑟𝑒𝑞𝑗𝑖
N
where, freqji is the frequency of NEG Gj in state Si such that i vary from integer 1 to N and j is constant. This
means i is a variable for different states and j is constant for Gj.
47

48. Proposed Definitions
Network Evolution Graphlets information is a doubleton set of retrieved NEGs (as a subgraphs)
and their aggregate frequencies Aggregate_freqj. The NEGs information is denoted as
NEGs_info = < Gj, Aggregate_freqj > | 0 ≤ j, mꞌ ≤ M
where, Gj is jth NEG with aggregate frequency as Aggregate_freqj and j is the enumeration of the
NEG; such that mꞌ is the number of retrieved NEGs (distinctly non-redundant graphlets) over all
the states in SS.
48

49. System State Complexity (SSC) &
Evolving System Complexity (ESC)
System State Complexity of Si (SSCi) =
σ𝑗=0
𝑚
(𝑓𝑟𝑒𝑞𝑗𝑖 ×𝐶𝑗)
σ𝑗=0
𝑚
𝑓𝑟𝑒𝑞𝑗𝑖
where Cj denotes cyclomatic complexity and freqji represents frequency for graphlet Gj of state Si
and m is count of retrieved graphlets. The SSCs_info denotes the SSCs information as
SSCs_info = < Si, SSCi > | 0 ≤ i ≤ N
Evolving System Complexity =
σ𝑗=0
𝑚ꞌ (𝐴𝑔𝑔𝑟𝑒𝑔𝑎𝑡𝑒_𝑓𝑟𝑒𝑞𝑗×𝐶𝑗)
σ𝑗=0
𝑚ꞌ 𝐴𝑔𝑔𝑟𝑒𝑔𝑎𝑡𝑒_𝑓𝑟𝑒𝑞𝑗
where, Aggregate_freqj denotes the aggregate frequency of a graphlet Gj over time, Cj denotes
the cyclomatic complexity of the graphlet Gj and mꞌ is count of retrieved NEGs.
49

50. Illustrative example
Subgraph mining for a state series
SS = {S1, S2, S3} represented as three
evolving networks.
50

51. Illustrative example
Complexity calculation for three states of an evolving system
For state S1 graphlets (G0, G2, and G71) has
◦ cyclomatic complexity (C0 = 1, C2 = 2, C71 = 1) and
◦ frequency (freq0,1 = 25, freq2,1 = 25, freq71,1 = 50).
SSC for S1 is calculated as
(25 × 1 + 25 × 2 + 50 × 1) ÷ (25 + 25 + 50) = 1.25
ESC for state series SS = {S1, S2, S3} is 1.816
51

52. Proposed Approach
52
System Network Complexity: An overview of the process-artifact interaction.

53. Proposed
Algorithms
53

54. 54
Proposed
Approach
System Network Complexity
An overview of flow-chart.

55. Experiments on Evolving Systems
Evolving Systems N
#
entities
Average #
neighbour
#
NEGs
ESC*
HDFS-Core1 15 3129 2.166 24 1.0054
Bible Translation2 13 246 1.456 17 1.456
Multi-sport Events3 13 141 1.786 10 1.00487
Frequent Market
Basket4 13 118 8.002 34 1.33888
Positive sentiment6
of movie genres5 16 284 2.661 4 1.03050
Negative sentiment6
of movie genres5 16 510 3.303 20 1.03683
55
M0, C0 = 1 M2, C2 = 2 M71, C71 = 1 M87, C87 = 1 M135, C135 = 1
M156, C156 =1 M178, C178=1 M185, C185=3 M192, C192=2 M198, C198 = 1

56. Experiments on Evolving Systems
56
Software Evolution Analytics on Evolving Software Systems:
➢ Hadoop – HDFS available on Software Repository

57. Experiments on Evolving Systems
57
Natural-Language Evolution Analytics on two Evolving Natural-Language systems:
➢ List of Bible Translation system available on Wikipedia
➢ List of Multi-sport events system available on Wikipedia

58. Market Evolution Analytics on Evolving Retail Market system:
➢ Frequent Market Basket system made from retail-market baskets available on UCI Repository
Experiments on Evolving Systems
58

59. Movie Evolution Analytics on two Evolving IMDb movie genre systems:
➢ Positive sentiment system made from IMDB movie-genre available on IMDb Repository
➢ Negative sentiment system made from IMDB movie-genre available on IMDb Repository
Experiments on Evolving Systems
59

60. Conclusions
The graphlets information is used to calculate the System State Complexity (SSC) for each state
Si.
The Network Evolution Graphlets(NEGs) information is used to calculate the Evolving System
Complexity (ESC) for a state series SS.
The SNC algorithm as a prototype SNC-Tool, which is used to conduct experiments on six open
internet based evolving systems. The tool
◦ retrieved network evolution subgraphs such as NEGs and Network Evolution Motifs (NEMs).
◦ calculated the complexity of individual states (as SSCs) and complexity of a state series (as ESC).
The SSCs are useful to compare between two states of an evolving system.
The ESC is useful to compare between two different evolving systems of the same domain.
60

61. System Evolution Recommender

62. Graph Evolution and Change Learning based
System Evolution Recommender
- Animesh Chaturvedi, Aruna Tiwari, and Shubhangi Chaturvedi. "SysEvoRecomd: Network Reconstruction by
Graph Evolution and Change Learning." IEEE Systems Journal 14.3 (2020): 4007-4014.
DOI: 10.1109/JSYST.2020.2988037
- Animesh Chaturvedi, and Aruna Tiwari. “SysEvoRecomd: Graph Evolution and Change Learning based System
Evolution Recommender”. 18th IEEE International Conference on Data Mining Workshops (ICDMW). Singapore,
2018, pp. 1499-1500. IEEE Computer Society DOI: 10.1109/ICDMW.2018.00217
- Animesh Chaturvedi, and Aruna Tiwari. “System Evolution Analytics: Deep Evolution and Change Learning of
Inter-Connected Entities”. 48th IEEE International Conference on Systems, Man, and Cybernetics (SMC), Miyazaki
Japan, October 2018, pp. 3075-3080. IEEE SMC Society DOI: 10.1109/SMC.2018.00657
62
• System Evolution Learning
• System Graph EvolutionLearning
• System Evolution Recommender
• Experiments on Evolving Systems

63. Deep Learning
Deep learning [22] form a Deep Neural Network (DNN)
DNN is good for deeper level learning based on classification of data with small error rate.
We used three famous techniques:
• Restricted Boltzmann Machines (RBM)
• Deep Belief Networks (DBN)
• denoising Autoencoders (dA)
These technique learns from the input matrixes to reconstruct an output matrix.
63
[22] LeCun Yann, Yoshua Bengio, and Geoffrey Hinton. “Deep learning.” Nature 521.7553 (2015): 436-444..

64. Definitions
Evolution and Change learning: evolution and change(s) happened to a time-variant data at
various states of a state series. Makes computer capable of evolution of a time-variant data
without explicit programming.
System Evolution Learning: evolution and change(s) happened over evolving system states at
various time points. Machine becomes capable of system evolution without explicit
programming.
Graph Evolution Learning: evolution and change(s) happened to evolving graphs as evolution
and change patterns. Machine becomes capable of graph evolution without explicit
programming. GEL -> Network Evolution Learning (NEL).
64

65. System Evolution Learning
The System Evolution Learning underlies upon graph structure learning as well as evolution and
change learning. Specifically, the SEL algorithm takes N+1 state of an evolving system (as input),
uses deep evolution learning and generates the SysNN (as output).
65
Algorithm System_Evolution_Learning(repository)
Retrieve N states (S1, S2… SN, SN+1) of an evolving system at a repository
1. ENN+1 = preprocess(repository)
2. MatrixesN+1 = convert(ENN+1)
➢ evolving_matrix = evolution_representor(MatrixesN)
3. graph_evolution_learning(evolving_matrix)
➢ binary_output_matrix = deep_evolution_learning(evolving_matrix)
4. binary_classifier_metric(binary_output_matrix, testing_matrix)

66. System Evolution Recommender
We proposed System Evolution Recommender (SysEvoRecomd) to do network reconstruction.
Preprocess a set of states to create a collection of N+1 graphs in the form of connection lists
with a mapping file.
Convert the collection of graphs into N+1 connection matrices.
The Graph Evolution and Change Learning (GECL) processes the N connection matrices and
produces an output matrix. The GECL uses Deep Evolution Learner (DEL) that do matrix
reconstruction to generate Deep System Neural Network (Deep SysNN). The GECL may also be
referred as Network Evolution and Change Learning (NECL).
Compare the output matrix with testing matrix (a matrix not used for training).
This makes the machine capable enough to understand the evolution in a state series without
any explicit programming.
66

67. SysEvoRecomd
67

68. Remodeling of Equations
68
RBM
DBN
DA
Autoencoder

69. System Graph Evolution Learning
69
We implemented the SEL algorithm in the tool as java application.
The tool learns evolution and change patterns from evolving matrix.
The tool requires machine enabled with JRE and JDK 7th version or higher.

70. Recommendation
70

71. Recommendation
71

72. Recommendation
72

73. Experiments on Evolving Systems
73
To do System Graph Evolution Learning, we extended three deep learning (RBM, DBN, and dA)
as three different deep evolution learning approaches.
For each evolving system, we used the three deep evolution learning approaches separately,
which resulted in three different SysNN.
We calculated their average referred as Average of Binary Metrics (ABM)
ABM =
accuracy + precision + fmeasure + recall
4

74. Experiments on Evolving Systems
74
Evolving
systems
Training
States (N)
Deep Evolution
learning
ABM
Hadoop HDFS-
Core
15
RBM 0.888
DBN 0.886
dA 0.883
Bible
Translation
13
RBM 0.649
DBN 0.749
dA 0.749
Multi-sport
Events
13
RBM 0.662
DBN 0.714
dA 0.696
Retail market
system
13
RBM 0.755
DBN 0.760
dA 0.737
Positive
sentiment of
movie genres
16
RBM 0.68
DBN 0.68
dA 0.680
Negative
sentiment of
movie genres
16
RBM 0.623
DBN 0.639
dA 0.631

75. Experiments on Evolving Systems
Further, our tool would be helpful in making recommendations about system evolution in
following ways.
▪ It can predict unknown connections between entities.
▪ It is helpful while upgrading system to a new state.
▪ It is helpful during intermediate phase of the system development.
▪ It can assist a system developer.
▪ It can speed-up the system development.
▪ It can do automatic correction of some errors during system development.
▪ It can determine possible future of the system.
75

76. Conclusions
This introduced a SysEvoRecomd algorithm that internally uses evolution and change learning.
This processes a series of graphs using deep evolution learning algorithm, which uses an
evolving matrix and outputs a SysNN.
An automated tool SysEvoRecomdTool did promising system network reconstruction by using
matrix reconstruction characteristics of remodeled deep learning.
This demonstrated the usefulness of intelligent recommendations using three variants of this
system learning approach based on RBM, DBN, and dA.
76

77. Service Evolution Analytics

78. Change and Evolution Mining of a
Distributed System
Animesh Chaturvedi, Aruna Tiwari, Shubhangi Chaturvedi, and Dave Binkley “Service Evolution Analytics:
Change and Evolution Mining of a Distributed System”, IEEE Transactions on Engineering Management,
Vol. 68, Issue No. 1, 2021. DOI:10.1109/TEM.2020.2987641
78
• Service Evolution Analytics
• Change mining of two versions: Service Change Classification
• Evolution mining of a Version series: Service Evolution Metrics
• Experiments

79. Service Evolution Analytics
- a service change classifier algorithm to assign change labels to a service’s operations and then extracts a
WSDL slice.
- four service evolution metrics to study cloud service evolution and to deduce facts from a version series
- We built tool (AWSCM) that supports change and evolution mining of an evolving distributed system.
79

80. Service Evolution Analytics
80
The Service Evolution Analytics model
Development and testing phases of the tool.
Starting at time t1 the development phase runs for time t2.
The subsequent testing phase starts at time t1 + t2 a runs for
time t3. Assuming the both phases end at time t1 + t2 + t3.

81. Change mining of two versions:
Service Change Classification
81

82. Change mining of two versions:
Service Change Classification
82

83. Change mining of two versions:
Service Change Classification
83

84. Change mining of two versions:
Service Change Classification
84
Change mining of two versions: Service Change Classification

85. 85
85
Change mining of two versions: Service Change Classification

86. Evolution mining of a Version series:
Service Evolution Metrics
86
Lines per operation in the WSDL
𝐿𝑂𝑊𝑆𝐷𝐿𝑖
=
𝑁𝑢𝑚𝑏𝑒𝑟 𝑜𝑓 𝑠𝑜𝑢𝑟𝑐𝑒 𝑙𝑖𝑛𝑒𝑠 𝑖𝑛 𝑡ℎ𝑒 𝑊𝑆𝐷𝐿 𝑓𝑜𝑟 𝑣𝑒𝑟𝑠𝑖𝑜𝑛 𝑖
𝑁𝑢𝑚𝑏𝑒𝑟 𝑜𝑓 𝑜𝑝𝑒𝑟𝑎𝑡𝑖𝑜𝑛𝑠 𝑖𝑛 𝑣𝑒𝑟𝑠𝑖𝑜𝑛 𝑖
Parameters per operation in the WSDL
𝑃𝑂_𝑖 =
𝑁𝑢𝑚𝑏𝑒𝑟 𝑜𝑓 𝑝𝑎𝑟𝑎𝑚𝑒𝑡𝑒𝑟𝑠 𝑖𝑛 𝑡ℎ𝑒 𝑊𝑆𝐷𝐿 𝑓𝑜𝑟 𝑣𝑒𝑟𝑠𝑖𝑜𝑛 𝑖
𝑁𝑢𝑚𝑏𝑒𝑟 𝑜𝑓 𝑜𝑝𝑒𝑟𝑎𝑡𝑖𝑜𝑛𝑠 𝑓𝑜𝑟 𝑣𝑒𝑟𝑠𝑖𝑜𝑛 𝑖
Messages per operation in the WSDL
𝑀𝑂𝑖 =
𝑁𝑢𝑚𝑏𝑒𝑟 𝑜𝑓 𝑚𝑒𝑠𝑠𝑎𝑔𝑒𝑠 𝑖𝑛 𝑡ℎ𝑒 𝑊𝑆𝐷𝐿 𝑓𝑜𝑟 𝑣𝑒𝑟𝑠𝑖𝑜𝑛 𝑖
𝑁𝑢𝑚𝑏𝑒𝑟 𝑜𝑓 𝑜𝑝𝑒𝑟𝑎𝑡𝑖𝑜𝑛𝑠 𝑓𝑜𝑟 𝑣𝑒𝑟𝑠𝑖𝑜𝑛 𝑖
Code Lines per operation in the WS code
𝑊𝑆𝐶𝐿𝑂_𝑖 =
𝑁𝑢𝑚𝑏𝑒𝑟 𝑜𝑓 𝑐𝑜𝑑𝑒 𝑙𝑖𝑛𝑒𝑠 𝑜𝑓 𝑊𝑆 𝑐𝑜𝑑𝑒 𝑖𝑛 𝑣𝑒𝑟𝑠𝑖𝑜𝑛 𝑖
𝑁𝑢𝑚𝑏𝑒𝑟 𝑜𝑓 𝑜𝑝𝑒𝑟𝑎𝑡𝑖𝑜𝑛𝑠 𝑖𝑛 𝑣𝑒𝑟𝑠𝑖𝑜𝑛 𝑖
LOWSDL ={(V1 , LOWSDL1 )… (Vi , LOWSDLi )… (VN , LOWSDLN )}
PO = {(V1 , PO1 )… (Vi , POi )… (VN , PON )}
MO = {(V1 , MO1 )… (Vi , MOi )… (VN , MON )}
WSCLO = {(V1 , WSCLO1 )… (Vi , WSCLOi )… (VN , WSCLON )}

87. Evolution mining of a Version series:
Service Evolution Metrics
87

88. Experiments
Web
services
DWSDL UWSDL Available at
SaaS Y Y Self-made
BookService Y Y Self-made
Eucalyptus Y Y GithHub
AWS Y N GithHub
88
Subset WSDL for Change analysis
Example for Eucalyptus-Cluster Controller (CC)
Feb-13.Eucalytpus-CC.wsdl ➔ Aug-13.Eucalytpus-CC.wsdl
Operation Change: ModifyNode inserted; MigrateInstances inserted.
Schema Change: CT ccInstanceType has modified; CT virtualBootRecordType has modified; CT
metricCounterType has modified; CT metricDimensionsType has modified.
May-15.Eucalytpus-CC.wsdl ➔ Dec-16.Eucalytpus-CC.wsdl
Operation Change: DescribePublicAddresses deleted; AttachNetworkInterface inserted;
DetachNetworkInterface inserted;
Schema Change: CT DetachVolumeResponseType inserted; CT ccInstanceType has modified;
CT netConfigType has modified;
Example for AWS-Elastic Compute Cloud (EC2)
Feb-13.ec2.wsdl ➔ Aug-13.ec2.wsdl
Operation Change: DescribeReservedInstancesModifications inserted; ModifyReservedInstances
inserted.
August-13.ec2.wsdl ➔ Oct-13.ec2.wsdl
Operation Change: AcceptVpcPeeringConnection inserted; CreateVpcPeeringConnection
inserted; DeleteVpcPeeringConnection inserted; DescribeVpcPeeringConnections inserted;
RejectVpcPeeringConnection inserted.
* where CT stands for ComplexType input-output data structure
* for inserted operations, it is obvious that their input-output data-structure were also inserted, thus we skipped its details.
* for deleted operations, it is obvious that their input-output data-structure were also deleted, thus we skipped its details.

89. 89

90. Conclusions
Approaches for change mining and evolution mining of an evolving distributed system.
• A service change classifier algorithm to assign change labels to a service’s operations and then extracts a
WSDL slice.
• Four service evolution metrics to study cloud service evolution and to deduce facts from a version
series.
A Service Evolution Analytics model and tool (AWSCM)
Four case studies to construct WSDL slices of four web services.
Demonstrated the service change classifier, our tools ability to reduce regression testing cost,
and service evolution metrics using two well-known cloud services: Eucalyptus-CC and AWS-EC2.
Helpful in subset regression testing, which in turn helps to maintain the QoS required by a SLA.
90

91. Thank You
Japanese
Hebrew
English
Merci
French
Russian
Danke
German
Grazie
Italian
Gracias
Spanish
Obrigado
Portuguese
Arabic
Simplified
Chinese
Traditional
Chinese
Tamil
Thai
Korean
https://sites.google.com/site/animeshchaturvedi07