The document discusses how software architecture is currently static and difficult to change, but emerging technologies could enable dynamic, self-adaptive systems. By combining infrastructure APIs, big data analytics, and a proposed "architecture API", systems could monitor and adjust their structure and behavior in response to changing conditions. This would allow systems to automatically scale resources, replace components, and reconfigure in an intelligent way. The speaker provides an example of a risk estimation system adapting its architecture based on analytics. Overall, the talk argues that cross-disciplinary research is needed to develop first-class runtime models of software architecture.

1. endava.com
QUALITY. PRODUCTIVITY. INNOVATION.
When Architecture Meets Data
Self-Aware Systems for an Imperfect World
Eoin Woods
IEEE Software Experts Summit
Beijing, October 2015

2. 2
When Architecture
Meets Data
Self-Aware Systems for an
Imperfect World
 Defining Architecture
 Architecture Today
 Where Architecture is Going
 Self-Adaptive Systems
 Examples
 Implications For Us All
QUALITY. PRODUCTIVITY. INNOVATION.

3. 3
3
To begin: what is “architecture”?
Many uses of the term “architecture”
For our purposes:
Software Architecture is the set of software related
elements that comprise a software-intensive
system, their relationships, and the set of design
decisions that allow the system to meet its system
quality requirements (“non-functionals”)
Software architecture tends to be the decisions that
are difficult to change once implemented [Fowler]
QUALITY. PRODUCTIVITY. INNOVATION.

4. 4
4
Architecture Today
Software architecture today can be characterised as:
• Static yet Ephemeral
• Set in stone, difficult to discern in the running system
• Depends on “Up-Front” Platform Design
• The least agile part is the runtime platform design
• Difficult to change
• Architectural change is a manual, time-consuming affair
QUALITY. PRODUCTIVITY. INNOVATION.

5. 5
5
Architecture – Static yet Ephemeral
QUALITY. PRODUCTIVITY. INNOVATION.
uml-diagrams.org
http://msdn.microsoft.com

6. 6
6
Architecture – Big Platform Up Front
QUALITY. PRODUCTIVITY. INNOVATION.
http://www.clipartpanda.com/
http://kenwoodrecords.com/
http://www.aberdeeninc.com/
http://www.tbngconsulting.com/

7. 7
7
Architecture – Difficult to Change
QUALITY. PRODUCTIVITY. INNOVATION.
http://pubs.opengroup.org/
https://www.thehouseshop.com

8. 8
8
Where Architecture is Going
Where do current trends point for architecture?
QUALITY. PRODUCTIVITY. INNOVATION.
Data Driven
Dynamic
http://daily.captaindash.com/

9. 9
9
Learning from Infrastructure
Infrastructure is undergoing a revolution
• Static, device centric, manual change, request based
… to dynamic, virtualised, automated, policy based
• Enabled by the “API for Infrastructure” – “SDx”
QUALITY. PRODUCTIVITY. INNOVATION.
http://www.cxotoday.com/story/cisco-vmware-deliver-next-generation-cloud-infrastructure/
http://www.bbc.co.uk/news/business-22879160

10. 10
10
Democratisation of Data
Data and analytics are also undergoing major change
• Centralised very expensive warehouses
… to scalable open source and SaaS analysis platforms
• Platform and algorithms becoming accessible commodities
• Allow predictive analytics on large data sets
QUALITY. PRODUCTIVITY. INNOVATION.
http://www.iconshut.com/
http://www.dbtekpro.com/2014/10/21/scaling-data-apps-cloud/

11. 11
11
Analytics Providing System Insight
Analytics possibilities
• Descriptive – what has happened so far?
• Predictive – what is likely in the future?
• Prescriptive – how do we optimise in response?
Example using Predictive Analytics
• Transaction profiling of system activity metrics
• Machine learning to “teach” the system what
peaks in activity look like
• Resulting model predicting future activity peaks
QUALITY. PRODUCTIVITY. INNOVATION.
https://azure.
microsoft.com

12. 12
12
Impact on Software Architecture?
What could the effect be on software architecture?
• SDx gives measurement and control of platform
• Big data allows insight into system operation
• SDx + Big Data == dynamic platform
• Where does this leave the software architecture?
• The weak link?
QUALITY. PRODUCTIVITY. INNOVATION.
Picture: infralayer – analytical controller –
Software architecture static on top

13. 13
13
Impact on Our Systems
• SDx + Analytics == dynamic platform
• Software architecture is still static – left behind?
QUALITY. PRODUCTIVITY. INNOVATION.
Dynamic
Infrastructure
Platform
Analytics
Based
Controller
Observations
Changes
http://msdn.microsoft.com

14. 14
14
• What if we had an API for our architecture?
• Needs first class architecture in the code
• SDI + Analytics + SDA == dynamic system
Architecture
API
Control
The Missing API - Architecture
QUALITY. PRODUCTIVITY. INNOVATION.
Platform
API
Analytics
Based
Controller
Control
Dynamic
Infrastructure
Platform
Measurements

15. 15
Not a new idea
1999 – IEEE Intelligent Systems Article
• An Architecture Based Approach to Self-Adaptive
Systems
• Oreizy, Gorlick, Taylor, Heimbigner, Johnson,
Medvidovic, Quilici, Rosenblum and Wolf
2009 – QoSA Conference theme
• Architectures for Self-Adaptive Software Systems
Research still going on today
• Recent survey paper “A survey on engineering
approaches for self-adaptive systems” by Krupitzer,
Roth, VanSyckel, Schiele, and Becker
Self-Adaptive Systems
QUALITY. PRODUCTIVITY. INNOVATION.

16. 16
16
Self-Adaptive Systems
QUALITY. PRODUCTIVITY. INNOVATION.
Evolution
Maintain Integrity
Make changes;
collect observations
Adaptation Evaluate
Operation
Plan Change
Apply
Change
Model
(Adapted from
Oreizy 99)
System

17. 17
17
Self-Adaptive Systems
QUALITY. PRODUCTIVITY. INNOVATION.
Conditional
Statements
Online
Algorithms
Generic
Algorithms
Algorithm
Selection
Genetic
Algorithms &
Machine
Learning
(Adapted from
Oreizy 99)

18. 18
18
What problems might this solve?
We could have systems that …
• Auto-scale reactively
• Add capacity when load increases (we have this today)
• Intelligently auto-scale predictively
• Adjust capacity for predicted peaks and troughs
• Change their functional behaviour
• Predictively replace processing components to meet
predicted requirements
QUALITY. PRODUCTIVITY. INNOVATION.

19. 19
19
Risk Estimator
An Example
QUALITY. PRODUCTIVITY. INNOVATION.
Transaction
Processor
Request
Handler
Demand
Message Bus
Risk Calculator
Risk Dashboard
Performance Analytics
Predictive
ControllerRisk Estimator
Risk Calculator

20. 20
20
An API for Architecture
Controlling our architecture would involve:
• Knowing the current structure (a model)
• Adding & removing elements and connectors
• Being able to “rewire” elements & connectors
• Constraints to reject invalid structures
• API implemented by the platform (PaaS)
Could extend the programming language or be an
external language – but in the code & “compiled”
• Maybe related to existing “ADLs”?
QUALITY. PRODUCTIVITY. INNOVATION.

21. 21
21
An API for Architecture
Our example using an imaginary API
QUALITY. PRODUCTIVITY. INNOVATION.
arch = getArchitecture("MyArch")
riskCalc = arch.getElement("RiskCalculator")
msgBus = calc.getConnector("inboundMsg")
dashBrd = calc.getConnector("dashboard")
riskEst = arch.getElement("RiskEstimator")
riskEst.setConnector("dashboard", dashBrd)
riskEst.setConnector("inboundMsg", msgBus)
riskCalc.setConnector("dashboard", null)
riskCalc.setConnector("inboundMsg", null)
riskCalc.deactivate()
riskEst.activate()
arch.commitChanges()

22. 22
22
Implications
What does this mean for research and practice?
• More cross disciplinary research than is normal
• Data, machine learning, adaptive systems, middleware, …
• More industry and academic cooperation
• Industry has the systems, academia the models
• First class runtime software architecture
• You need a model of the architecture to change it
• No (general) runtime architecture models in use today
• The challenge Software Architecture research needs?
QUALITY. PRODUCTIVITY. INNOVATION.

23. 23
23
Our Future Systems Architectures
QUALITY. PRODUCTIVITY. INNOVATION.
Handler
Processor Calculator
Store
SDx Infrastructure System Analytics Architecture API Self-Adaptive Systems
Dynamic
Data-Driven
Systems

24. 24
Thank you
QUALITY. PRODUCTIVITY. INNOVATION.
Eoin Woods
Endava
eoin.woods@endava.com
+44 207 367 1000
en_ewoods