(Invited talk at IT Circle Netherlands) Many organizations struggle with the challenge of sustainability and how you can bring it to your IT organization. Patricia Lago, professor at VU Amsterdam is leading the Software and Services research group at VU. Her research focus is on energy-efficient software engineering and software sustainability. She, together with her colleague Ivano Malavolta shares her thoughts and experiences on how to deal with this emerging topic.

1. Patricia Lago & Ivano Malavolta
Vrije Universiteit Amsterdam
Presented at: IT Circle 2021
How to bring
Sustainability in your
Organization – Green IT
“Architecting for sustainability”

2. The S2 group @ VU Amsterdam
I/O Magazine, Issue 3, Dec. 2020
s2group.cs.vu.nl

3. The S2 group @ VU Amsterdam
• Software Architecture Design for Digital Sustainability
• Architectural Technical Debt
• Software Energy Footprint
• Robotics Software
• Software Engineering and Ethics
• Software Adaptation
s2group.cs.vu.nl

4. Collaboration opportunities in
Higher education, research- and professional training
MASTER'S TRACK IN
COMPUTER SCIENCE
SOFTWARE
ENGINEERING
AND GREEN IT
WWW.VU.NL/COMPUTERSCIENCE

5. Collaboration opportunities in
Research
GREEN LAB
SUSTAINABILITY
ASSESSMENT
FRAMEWORK
s2group.cs.vu.nl

6. “LIKE PERFORMANCE, RELIABILITY, SECURITY,
SUSTAINABILITY DOES NOT JUST HAPPEN
UNLESS WE PLAN FOR IT.”

7. IDENTIFY
VALUE
GATHER
AWARENESS
DEFINE
STRATEGY
SETUP
INFORMED
DECISION
MAKING
The journey toward digital sustainability
APPOINT
CHAMPIONS
AND
WARRIORS
THE GREEN
LAB
Energy
efficiency in
DATACENTERS
Build successful & repeatable green practices
How to bring Sustainability in your Organization
Energy
efficiency in
MOBILE WEB
APPS
Energy
efficiency in
ROBOTICS

8. IDENTIFY
VALUE
GATHER
AWARENESS
DEFINE
STRATEGY
SETUP
INFORMED
DECISION
MAKING
The journey toward digital sustainability
[national and international sources]
APPOINT
CHAMPIONS
AND
WARRIORS

9. Why should I do it?
MARKET
SHARE
BELIEFS I HAVE TO
IDENTIFY
VALUE
GATHER
AWARENESS
DEFINE
STRATEGY
SETUP
INFORMED
DECISION
MAKING
APPOINT
CHAMPIONS
AND
WARRIORS

10. Why should I do it?
MARKET
SHARE
BELIEFS I HAVE TO
IDENTIFY
VALUE
GATHER
AWARENESS
DEFINE
STRATEGY
SETUP
INFORMED
DECISION
MAKING
APPOINT
CHAMPIONS
AND
WARRIORS
ADDED VALUE FOR THE BUSINESS + INTRINSIC VALUES THE COMPANY BELIEVES IN

11. The Horizon of Sustainable Digital Infrastructure Scenarios
Data
traffic
Energy
aware
Software
Flexible
Distributed
IT + Energy sectors
Flexible
Disaggregated
Dynamic
Prioritization
Verdecchia, R., Lago, P., & de Vries, C. (2021). The LEAP Technology Landscape: Lower Energy Acceleration Program
(LEAP) Solutions, Adoption Factors, Impediments, Open Problems, and Scenarios. https://tinyurl.com/leap-tech-landscape
LEGEND:
- Promise
- Characteristic
EXAMPLE: A CLOUD-BASED SOCIETY

12. IDENTIFY
VALUE
GATHER
AWARENESS
DEFINE
STRATEGY
SETUP
INFORMED
DECISION
MAKING
APPOINT
CHAMPIONS
AND
WARRIORS
What does it mean?
A SUSTAINABLE ORGANIZATION
“an airline company” Example (2021)
Lago, P. (2019). Architecture design decision maps for software
sustainability. In IEEE/ACM 41st International Conference on Software
Engineering: Software Engineering in Society, https://tinyurl.com/DesDmaps

13. IDENTIFY
VALUE
GATHER
AWARENESS
DEFINE
STRATEGY
SETUP
INFORMED
DECISION
MAKING
APPOINT
CHAMPIONS
AND
WARRIORS
What does it mean?
A SUSTAINABLE ORGANIZATION
“an airline company” Example (2021)

14. What are the main Open Problems hindering action
Lack of policies, KPIs,
ACTIVATING STRATEGIES
Need for COORDINATED
CHANGE, and leading
champions
Lack of KNOWLEDGE and awareness
Verdecchia, R., Lago, P., & de Vries, C. (2021). The LEAP Technology Landscape: Lower Energy Acceleration Program
(LEAP) Solutions, Adoption Factors, Impediments, Open Problems, and Scenarios. https://tinyurl.com/leap-tech-landscape

15. IDENTIFY
VALUE
GATHER
AWARENESS
DEFINE
STRATEGY
SETUP
INFORMED
DECISION
MAKING
APPOINT
CHAMPIONS
AND
WARRIORS
What does it mean?
What is software sustainability? Why architecture?

16. Perspectives on Sustainability
Dimensions Order of effects
UN Sustainable
Development Goals
DIRECT IMPACT
(technology)
ENABLING IMPACT
(supported processes)
SYSTEMIC IMPACT
(change in behavior)
https://sdgessentials.org/why-
the-world-needs-the-sdgs.html
https://tinyurl.com/y3cfecs4
REBOUND EFFECTS
(negate intended impact)

17. What is software sustainability?
Direct impact
(sustainable software)
Inward looking
SOFTWARE-INTENSIVE
SYSTEM
ENERGY EFFICIENCY MAINTAINABILITY
STABILITY
Indirect impact
(software for sustainability)
Outward looking
SOFTWARE-INTENSIVE
SYSTEM
TRUSTABILITY
USABILITY
DIGITAL INCLUSIVENESS

18. Lago, P. © 2016-2018 SoSA: A Software Sustainability
Assessment Method. Available at: https://goo.gl/HuY6tf
Condori-Fernandez, N., & Lago, P. (2018). Characterizing
the contribution of quality requirements to software
sustainability, Journal of Systems and Software, 137, 289-305.
Why software architecture? choosing the dimensions of focus
(Software) sustainability is cross-dimensional
Lago, P. et al (2020). Designing for Sustainability: Lessons
Learned from Four Industrial Projects. EnviroInfo.
https://tinyurl.com/fourcases
AI-assisted desk workers

19. Lago, P. © 2016-2018 SoSA: A Software Sustainability
Assessment Method. Available at: https://goo.gl/HuY6tf
Why software architecture? choosing the dimensions of focus
(Software) sustainability is cross-dimensional
Lago, P. et al (2020). Designing for Sustainability: Lessons
Learned from Four Industrial Projects. EnviroInfo.
https://tinyurl.com/fourcases
AI-assisted desk workers
Condori-Fernandez, N., & Lago, P. (2018). Characterizing
the contribution of quality requirements to software
sustainability, Journal of Systems and Software, 137, 289-305.

20. (Software) sustainability is measurable only over time
Why software architecture? making decisions that last
Lago, P. © 2016-2018 SoSA: A Software Sustainability
Assessment Method. Available at: https://goo.gl/HuY6tf
Lago, P. et al (2020). Designing for Sustainability: Lessons
Learned from Four Industrial Projects. EnviroInfo.
https://tinyurl.com/fourcases
AI-assisted desk workers
Condori-Fernandez, N., & Lago, P. (2018). Characterizing
the contribution of quality requirements to software
sustainability, Journal of Systems and Software, 137, 289-305.

21. (Software) sustainability is measurable only over time
Why software architecture? making decisions that last
Lago, P. © 2016-2018 SoSA: A Software Sustainability
Assessment Method. Available at: https://goo.gl/HuY6tf
Condori-Fernandez, N., & Lago, P. (2018). Characterizing the
contribution of quality requirements to software sustainability, Journal
of Systems and Software, 137, 289-305.
Lago, P. et al (2020). Designing for Sustainability: Lessons
Learned from Four Industrial Projects. EnviroInfo.
https://tinyurl.com/fourcases
AI-assisted desk workers

22. What does it mean for me?
IDENTIFY
VALUE
GATHER
AWARENESS
DEFINE
STRATEGY
SETUP
INFORMED
DECISION
MAKING
APPOINT
CHAMPIONS
AND
WARRIORS
• A sustainability goal à one or more
questions each regarding a
sustainability-quality (SQ) concern
• …

23. What does it mean for me?
IDENTIFY
VALUE
GATHER
AWARENESS
DEFINE
STRATEGY
SETUP
INFORMED
DECISION
MAKING
APPOINT
CHAMPIONS
AND
WARRIORS
WhatsApp
Neighborhood
Prevention
1: the original plan
2: the measured impact 3: the corrected plan

24. How can I build true impact?
IDENTIFY
VALUE
GATHER
AWARENESS
DEFINE
STRATEGY
SETUP
INFORMED
DECISION
MAKING
APPOINT
CHAMPIONS
AND
WARRIORS
• A sustainability goal à one or more
questions each regarding a
sustainability-quality (SQ) concern
• The measures of the SQ concerns
express the impact
• The variation (or trend) over time
expresses the effect towards
the sustainability goal

25. Decisions based on a (quantifiable) informed strategy
Renewable Energy Provisioning: Sharing IoT assets
[Vandebron]
Sustainability Goal = {
• Balance Energy prosumption
• Maximize Trust
• Enable positive behavioral
change }
Lago, P. et al (2020). Designing for Sustainability: Lessons
Learned from Four Industrial Projects. EnviroInfo.
https://tinyurl.com/fourcases

26. Condori-Fernandez, N., Lago, P., Luaces, M. R., & Places, Á. S. (2020). An action research for improving the sustainability assessment framework
instruments. Sustainability (Switzerland), 12(4), 1-25. https://doi.org/10.3390/su12041682
A Checklist to help define the
elements of a DM
The Decision Map (DM) to help explore
the design space and make decisions
The SQ dependency matrix to
help identify the dependencies
among DM elements
The Decision Graph to help
assign the right impact
timescale to DM elements
The Sustainability-Quality (SQ) Model to
define concerns and measures
The SAF Toolkit

27. Who will make it possible?
IDENTIFY
VALUE
GATHER
AWARENESS
DEFINE
STRATEGY
SETUP
INFORMED
DECISION
MAKING
APPOINT
CHAMPIONS
AND
WARRIORS
champions warriors
communication
skills
competences
Images: @ Plants Vs. Zombies Garden Warfare 2

28. Build successful & repeatable green practices
THE GREEN
LAB
Energy
efficiency in
DATACENTERS
Energy
efficiency in
MOBILE APPS
Energy
efficiency in
ROBOTICS

29. The Green Lab
THE GREEN
LAB
Energy
efficiency in
DATACENTERS
Energy
efficiency in
MOBILE WEB
APPS
Energy
efficiency in
ROBOTICS
Students measure real
software products
Our infrastructure for
experimenting
on software
● energy efficiency
● performance
● ...
Industry-driven experiments
A MASTER COURSE
A PLATFORM
A COLLABORATION PLATFORM

30. The Green Lab
Green cluster
Promox (Debian) vmWare OS vmWare OS
Wattsup Wattsup Wattsup
MobileServerVM2
192.168.188.32
Ubuntu 16.04.1 LTS
.17 - androidApps
Ubuntu 16.04.1 LTS
.12 - emabot
Ubuntu 16.04.1 LTS
HD: 800Gb, RAM: 36Gb,
CPU: 2.4Ghz x 4 cores HD: 1Tb, RAM: 18Gb, CPU: 2.00Ghz
HD: 126Gb, RAM: 64Gb,
CPU: 2.50Ghz
(VM management server) (SuperMicro STF)
(Windows server)
.9 - vSphere web client
vmWare OS Windows Desktop Server 2007 Ubuntu 16.04.1 LTS
vRealize
vSphere client
Server for the web-client
Wattsup manager
Other utilities
STF ADB
vmWare OS
HD: 1Tb, RAM: 16Gb,
CPU: 2.00Ghz x 2 cores
HD: 70Gb, RAM: 8Gb,
CPU: 2.33Ghz x 2 cores
HD: 1Tb, RAM: 32Gb,
CPU: 3.4Ghz x 4 cores
.19 - franz2017
Ubuntu 16.04.1 LTS
HP DL360 G7 HP DL380 G5 System x3550 M4
HP DL380 G5 ---
SuperMicro 813M-4
(management)
Promox (Debian)
HD: 36Tb, RAM: 192Gb,
CPU: 2 x (2.6Ghz x 4 cores)
SuperMicro Superserver
.23 - Tanjina
Ubuntu 18.04.2 LTS
.20 - Roberto
Ubuntu 18.04.2 LTS
.21 - KishanNirghin
Ubuntu 18.04.2 LTS
.22 - Ivano
Ubuntu 18.04.2 LTS
.29 - Nick
Ubuntu 18.04.2 LTS
.25 - Katerina
Ubuntu 18.04.2 LTS
.26 - Emitza
Ubuntu 18.04.2 LTS
.28 - Covid-19
Ubuntu 18.04.2 LTS
.27 - Ilias
Ubuntu 18.04.2 LTS
.30 -
FrancescoOsbourne
Ubuntu 18.04.2 LTS
.31 - Eoin
Ubuntu 18.04.2 LTS
USB/Wifi

31. Measurement equipment (1)
● Hardware Watt meter
● Measures instantaneous
power
○ Logs it via USB1
○ Via Python or C utilities
● 60 Hz frequency
● Highly used in the
literature2
Watts up Pro
1 https://github.com/yyongpil/wattsup
2 https://scholar.google.it/scholar?q=%22Watts+up%3F+Pro%22&hl=it&as_sdt=0,5
Image from: https://doi.org/10.1016/j.jss.2018.07.077

32. Measurement equipment (2)
● Bypass the battery of the
device
● Portable
● Measures current
○ High accuracy
○ 5KHz frequency
○ Logs it via USB
○ Python-based API
Monsoon power Monitor1
1 https://www.msoon.com/online-store/High-Voltage-Power-Monitor-p90002590

33. Smartphone Test Farm (STF)
https://github.com/DeviceFarmer/stf
● Debug/control several
mobile devices remotely
● Web-based UI
● Real-time screen view
● Execute remote shell
commands
● Manage device inventory

34. Software for orchestrating experiments
A framework to automatically execute measurement-
based experiments on native and web apps
Android
Runner
Intuition - users define their experiment in a descriptive fashion, and
then its execution is fully automatic, customizable, and replicable

35. How does it work?

36. Experiment configuration

37. Benchmarking apps

38. Examples of results

39. Energy efficiency in
DATACENTERS
THE GREEN
LAB
Energy
efficiency in
DATACENTERS
Energy
efficiency in
MOBILE WEB
APPS
Energy
efficiency in
ROBOTICS
Roberto Verdecchia, Giuseppe Procaccianti, Ivano Malavolta, Patricia Lago, Joost Koedijk. Estimating Energy
Impact of Software Releases and Deployment Strategies: the KPMG Case Study. In Proceedings of the 11th
ACM/IEEE International Symposium on Empirical Software Engineering and Measurement, ESEM 2017
Partner:

40. Research questions
• .NET Platform supporting Governance,
Risk and Compliance (GRC) processes.
• Thousands of users worldwide
• ~11.000 eLOCs
• Personalized questionnaires
• Dashboard and reports for results overview
The KPMG case
THE GREEN
LAB
Energy
efficiency in
DATACENTERS
Energy
efficiency in
MOBILE WEB
APPS
Energy
efficiency in
ROBOTICS

41. Collaboration between academia and
industry
We want to consume
less energy in our datacenter
Introduction to
Qubus
We are interested in: releases
and deployment strategies
Validation of general
characteristics
Qubus 6.2.3 or Qubus 6.5.7?
Centralized or Distributed?
Empirical
experiment
Clarifications and
refinements
Results
discussion
Preliminary
experiment
definition
The KPMG case
THE GREEN
LAB
Energy
efficiency in
DATACENTERS
Energy
efficiency in
MOBILE WEB
APPS
Energy
efficiency in
ROBOTICS

42. Research questions
How can we improve the energy efficiency of Qubus?
• Which release is more energy efficient?
• Qubus 6.2.3 VS Qubus 6.5.7
• Which deployment strategy is more energy efficient?
• Centralized VS Distributed
VS
THE GREEN
LAB
Energy
efficiency in
DATACENTERS
Energy
efficiency in
MOBILE WEB
APPS
Energy
efficiency in
ROBOTICS

43. Infrastructure
THE GREEN
LAB
Energy
efficiency in
DATACENTERS
Energy
efficiency in
MOBILE WEB
APPS
Energy
efficiency in
ROBOTICS

44. Old - C Old - D New - C New - D
Results
Scenario 1
Old - C Old - D New - C New - D
Scenario 2
Know precisely the usage scenarios of your
software applications!
Using a higher number of machines does NOT
always imply higher energy consumption
Energy
consumption

45. THE GREEN
LAB
Energy
efficiency in
DATACENTERS
Energy
efficiency in
MOBILE WEB
APPS
Energy
efficiency in
ROBOTICS
Energy efficiency in
MOBILE WEB APPS
Kwame Chan Jong Chu, Tanjina Islam, Miguel Morales Exposito, Sanjay Sheombar, Christian Valladares, Olivier
Philippot, Eoin Martino Grua, Ivano Malavolta. Investigating the correlation between performance scores and
energy consumption of mobile web apps. In Proceedings of the International Conference on Evaluation and
Assessment on Software Engineering (EASE), pp. 190–199, 2020
Partner:

46. Motivation
Performanc
e Analysis
Tool
Mobile
Web App
Input
Low Scoring?
(Low performance)
Development Strategy
Adoption of Best
Practices
Performance
Score
Improvement
Guidelines
Output
Output
● Mobile is a dominant segment of
internet traffic
● Perceived Performance is critical to
user engagement
● Performance Analysis tools help
Adoption of best practices

47. Motivation
Performanc
e Analysis
Tool
Mobile
Web App
Input
● Tooling to help developers assess
energy efficiency AND adopt best
practices is lacking
● Energy Inefficiency can induce user
abandonment
● There are tools to measure energy
consumption, but do not necessarily
provide best practices to help
developers
Our Research:
● Tries to bridge the Gap to support
developers in adopting best
practices
● Does there exist a relationship
between Performance Scores and
Energy Consumption?
Energy
Consumption
Analysis Tool
Improvement
Guidelines
?
Measured Energy
Consumption
I
n
p
u
t
Output
Development Strategy
Output
Output
Improvemen
t Guidelines
Performance
Score

48. Research question
To what extent do performance scores correlate to
the energy consumption of mobile web apps?

49. Study design
Independent Variable:
Performance Level
(Good, Average, Poor)
Controlled By:
Selection of web-apps per
level via scoring.
Dependent Variable:
Energy Consumption
Upon Loading Web-App
on Mobile Browser (Joules)

50. Experiment Execution
Top 100 Web Sites Get Lighthouse
Performance
Scores
Performance Scores Generation
For each
Min:
0.01
Mean
: 0.52
Median:
0.57
Max:
1.0
Good
100 - 75
Average
74 - 45
Poor
44 -
0
Subjects Selection
21 REAL Web Apps
(7 per Category)
Per Stratified Random Sample

51. Experiment Execution
Orchestrator
Energy Assessment
Energy Profiler & Orchestrator
Orchestration
25 Iterations per App
HTC Nexus 9
Precondition
● Calibration providing
probe trust level
● Reference Power
Consumption (OS
Consumption)
Execute Tests
(UIAUtomator)
Sanitize OS &
Configuration
Clearing Device
Caches
Tail Energy
Mitigation
Energy Consumption values are
collected in Watts:

52. Results
Boxplot Performance level vs Energy Consumption
Density curve for energy consumption

53. THE GREEN
LAB
Energy
efficiency in
DATACENTERS
Energy
efficiency in
MOBILE WEB
APPS
Energy
efficiency in
ROBOTICS
Energy efficiency in
ROBOTICS
Ivano Malavolta, Katerina Chinnappan, Stan Swanborn, Grace Lewis, Patricia Lago. Mining the ROS ecosystem
for Green Architectural Tactics in Robotics and an Empirical Evaluation. In Proceedings of the 18th International
Conference on Mining Software Repositories (MSR), 2021.
Partner:

54. The software of robotics systems is
getting more and more large and
complex
Robotics systems are central to
many industrial sectors
Robotics systems
consume a large amount
of energy
https://www.themanufacturer.com/articles/slam-software-
powering-next-generation-autonomous-industrial-robots/

55. Goal
To identify and
empirically evaluate
architectural tactics for
energy-efficient robotics software
By mining the ROS ecosystem to identify
the green tactics
By conducting an experiment on a real robot
for evaluating the identified green tactics
How

56. Green tactics identification
https://github.com/S2-group/msr-2021-robotics-
green-architectural-tactics-replication-package
Replication package

57. Empirical evaluation of the green tactics
Name Treatments
Tactic - Baseline (no tactics
applied)
- Tactic EE1 applied
- Tactic EE2 applied
- Tactic EE3 applied
- Tactic EE4 applied
- All tactics applied
Movement - No movement
- Fixed movement
- Autonomous movement
Environme
nt
- Empty
- Obstacles
Independent variables
30 trials
10 Runs per trial
300 runs = ~10 Hours
Dependent variable
energy consumption in J during the whole mission
Arduino Nano + INA 219 sensor
Mission
To move at 0.6m/s within the arena for 2 minutes
while:
(1) continuously video-recording at 60 FPS and
(2) stopping and doing a 360° rotation every 20s

58. (Some) Results
1. On average, the application of the
green tactics improve energy
efficiency
a. However, not always with the
same magnitude
2. The combination of all tactics (C)
improves energy efficiency more
than each tactic in isolation
1. Green tactics reduce energy
consumption also across different
movements and environments

59. To summarize …
The Green Lab
THE GREEN
LAB
Energy
efficiency in
DATACENTERS
Energy
efficiency in
MOBILE WEB
APPS
Energy
efficiency in
ROBOTICS
Students measure real
software products
Our infrastructure for
experimenting
on software
● energy efficiency
● performance
● ...
Industry-driven experiments
A MASTER COURSE
A PLATFORM
A COLLABORATION PLATFORM
Energy efficiency in
DATACENTERS
THE GREEN
LAB
Energy
efficiency in
DATACENTERS
Energy
efficiency in
MOBILE WEB
APPS
Energy
efficiency in
ROBOTICS
Roberto Verdecchia, Giuseppe Procaccianti, Ivano Malavolta, Patricia Lago, Joost Koedijk. Estimating Energy
Impact of Software Releases and Deployment Strategies: the KPMG Case Study. In Proceedings of the 11th
ACM/IEEE International Symposium on Empirical Software Engineering and Measurement, ESEM 2017
Partner:
THE GREEN
LAB
Energy
efficiency in
DATACENTERS
Energy
efficiency in
MOBILE WEB
APPS
Energy
efficiency in
ROBOTICS
Energy efficiency in
MOBILE WEB APPS
Kwame Chan Jong Chu, Tanjina Islam, Miguel Morales Exposito, Sanjay Sheombar, Christian Valladares, Olivier
Philippot, Eoin Martino Grua, Ivano Malavolta. Investigating the correlation between performance scores and
energy consumption of mobile web apps. In Proceedings of the International Conference on Evaluation and
Assessment on Software Engineering (EASE), pp. 190–199, 2020
Partner:
THE GREEN
LAB
Energy
efficiency in
DATACENTERS
Energy
efficiency in
MOBILE WEB
APPS
Energy
efficiency in
ROBOTICS
Energy efficiency in
ROBOTICS
Ivano Malavolta, Katerina Chinnappan, Stan Swanborn, Grace Lewis, Patricia Lago. Mining the ROS ecosystem
for Green Architectural Tactics in Robotics and an Empirical Evaluation. In Proceedings of the 18th International
Conference on Mining Software Repositories (MSR), 2021.
Partner:

60. To summarize …

61. Thank you
Credits: slides, ideas and results are a collective effort with my
bright and energetic colleagues in the S2 Group @Vrije
Universiteit Amsterdam s2group.cs.vu.nl
Download our papers from the S2 VU Research Portal
Check out what we teach: s2group.cs.vu.nl/pages/courses