Actually using metrics for making meaninful decisions Several development teams have metrics and analysis tools available that can be used to support important decisions, but very few of them actually use this information. This talk will introduce software analytics, a technique that can be used to use this information to make decisions and take action. It will also present a Lean Software Analytics Canvas, a simple tool that can be used by small and medium agile teams to incorporate software analytics in their development process. Realistic examples and results based on the experience of applying the canvas in real teams will be also presented.

1. A Lean Software
Analytics Canvas for
Agile Small Teams
Actually using
metrics for
meaningful
decisions
Eduardo Guerra
SFSCon 2021

3. I think the clients
would like more the
user interface if we
change it.
I think we will
have more clients
and we need
another server.

4. I think this algorithm
can be a problem for
performance and we
need to optimize it.
I think this part
of the code
needs urgent
refactoring.

5. How
development
team make
decisions Based on our
experience!

6. Based on our
experience!
The
experience
How
development
team make
decisions

7. I THINK For the resources
that you are going
to spend doing
that, you need to
be sure!

12. But what is
the question?

13. Finding what
metrics can help to
solve the project
problems.
Figuring out what I
can do with the
metrics from tools
that I have installed.

14. meet analytics

15. Analytics
“Analytics is a data-driven
approach to support decision
making widely used in marketing
in order to reach and better
understand customers.”
(Danvenport, 2009)

16. Software Analytics
“Software Analytics is to enable
software practitioners to perform data
exploration and analysis in order to
obtain insightful and actionable
information for data-driven tasks
around software and services.”
(Zhang et al., 2011)

17. to obtain
insightful and
actionable
information

18. Development practices
Testing methods
Bug treatment
Teams and collaboration
Productivity
Reuse of components
Customer and requirements
Maintenance and software evolution
Evaluating quality
Software usage
Services

19. Development Tools
Bug Reports
Application Data
Code Repository
Running Software
Source code
Documentation

20. Every measurement should be
done for a reason.
The entire team need to be aware
of the analytics process.
The project cannot be stopped just
to implement analytics.
The result should be decisions and
actions, and not numbers.

21. How to make
that suitable
for small and
medium size
teams?

23. goal
metric
decision
Things are spread all
over the development
tools and documents!

24. Software
Analytics Canvas

26. Example
Software Architecture

27. Why are
sometimes
reports take too
long to be
generated by
the software?

28. Why are
sometimes
reports take too
long to be
generated by
the software? Request Rates
Response Time
CPU usage

29. Why are
sometimes
reports take too
long to be
generated by
the software? Request Rates
Response Time
CPU usage
(a) start and end
time logs for report
execution.
(b) a tool to
monitor CPU usage.

30. Why are
sometimes
reports take too
long to be
generated by
the software? Request Rates
Response Time
CPU usage
(a) start and end
time logs for report
execution.
(b) a tool to
monitor CPU usage.
Configure tool to
monitor the CPU on the
server.
Implement request logs
into report feature.
Analyze runtime logs.

31. Why are
sometimes
reports take too
long to be
generated by
the software? Request Rates
Response Time
CPU usage
(a) start and end
time logs for report
execution.
(b) a tool to
monitor CPU usage.
Configure tool to
monitor the CPU on the
server.
Implement request logs
into report feature.
Analyze runtime logs.

32. Why are
sometimes
reports take too
long to be
generated by
the software? Request Rates
Response Time
CPU usage
(a) start and end
time logs for report
execution.
(b) a tool to
monitor CPU usage.
Configure tool to
monitor the CPU on the
server.
Implement request logs
into report feature.
Analyze runtime logs.
A large delay
when multiple
reports are
requested at the
same time.

33. Why are
sometimes
reports take too
long to be
generated by
the software? Request Rates
Response Time
CPU usage
(a) start and end
time logs for report
execution.
(b) a tool to
monitor CPU usage.
Configure tool to
monitor the CPU on the
server.
Implement request logs
into report feature.
Analyze runtime logs.
A large delay
when multiple
reports are
requested at the
same time.

34. Why are
sometimes
reports take too
long to be
generated by
the software? Request Rates
Response Time
CPU usage
(a) start and end
time logs for report
execution.
(b) a tool to
monitor CPU usage.
Configure tool to
monitor the CPU on the
server.
Implement request logs
into report feature.
Analyze runtime logs.
A large delay
when multiple
reports are
requested at the
same time.
CPU usage is
extremely high
when reports
are requested
at the same
time

35. Why are
sometimes
reports take too
long to be
generated by
the software? Request Rates
Response Time
CPU usage
(a) start and end
time logs for report
execution.
(b) a tool to
monitor CPU usage.
Configure tool to
monitor the CPU on the
server.
Implement request logs
into report feature.
Analyze runtime logs.
A large delay
when multiple
reports are
requested at the
same time.
CPU usage is
extremely high
when reports
are requested
at the same
time
When a report
takes time to
process, the
user request it
again,
overloading the
system.
Reports does
not need an
immediate
return.

36. Why are
sometimes
reports take too
long to be
generated by
the software? Request Rates
Response Time
CPU usage
(a) start and end
time logs for report
execution.
(b) a tool to
monitor CPU usage.
Configure tool to
monitor the CPU on the
server.
Implement request logs
into report feature.
Analyze runtime logs.
A large delay
when multiple
reports are
requested at the
same time.
CPU usage is
extremely high
when reports
are requested
at the same
time
When a report
takes time to
process, the
user request it
again,
overloading the
system.
Reports does
not need an
immediate
return.
CPU usage
should be
lower than
90%.
Make
processing of
reports
asynchronous to
avoid CPU
overhead.

37. Why are
sometimes
reports take too
long to be
generated by
the software? Request Rates
Response Time
CPU usage
(a) start and end
time logs for report
execution.
(b) a tool to
monitor CPU usage.
Configure tool to
monitor the CPU on the
server.
Implement request logs
into report feature.
Analyze runtime logs.
A large delay
when multiple
reports are
requested at the
same time.
CPU usage is
extremely high
when reports
are requested
at the same
time
When a report
takes time to
process, the
user request it
again,
overloading the
system.
Reports does
not need an
immediate
return.
CPU usage
should be
lower than
90%.
Make
processing of
reports
asynchronous to
avoid CPU
overhead.
Implement report
queue.
Implement CPU alert on
server.

38. The Lean Analytics Canvas
helps to reason about the
how the metrics can be used!

39. Software
Startup
Case studies introduced successfully the
use of the canvas in development teams!
Multinational
Company

41. eduardo.guerra@unibz.it