It's all about measuring, and that's just the easy part. The hard part is focusing on what really matters. I'll show you a bunch of metrics at different levels and from different perspectives, trying to make some sense out of them. I'll try to expand the current vision of what could be a metric, how it affects you and how you can choose your next ones. The main goal is being able to understand what you need to monitor and improve in order to get better at what you're doing.

1. IT Metrics in Real Life
IMPROVING YOURSELF BY MEASURING RIGHT

2. Hello!
I AM FABIO CICERCHIA
Chief Technology Officer @ Skuola.net
You can find me at: @fabiocicerchia

3. Let’s Clarify ‘em First

4. What’s a Metric?
Metrics are parameters or measures of quantitative assessment used
for measurement, comparison, or to track performance or production.
https://www.investopedia.com/terms/m/metrics.asp

5. Are they useful?
● Provide Transparency
● Highlight Improvements, Regressions or Mistakes
● Lead to Self Improvement
● Drive Decisions
● Reduce Costs
● Allow you to be Objective

6. What’s a KPI?
A KPI is a measurable value that demonstrates how effectively a
company is achieving key business objectives.
Organisations use KPIs at multiple levels to track performance measures.
https://www.klipfolio.com/resources/articles/what-is-a-key-performance-indicator

7. Are they useful?
● I’ll leave it to you

8. Recommendation #1
Measure what matters while it matters
https://www.oreilly.com/ideas/ask-the-cto-measuring-team-performance

9. Recommendation #2
The focus should be on the goal not the metric

10. Recommendation #3
Don't use raw metrics as performance evaluation

11. An Example By The Book

12. McCalls’ Software Quality Factors
https://www.researchgate.net/profile/Caroline_Jay/publication/264239621/figure/fig1/AS:614318999625766@1523476471560/McCalls-Software-Quality-Factors.png

13. - Auditability – The ease with which conformance to standards can be checked.
- Accuracy – The precision of computation and control.
- Communication Commonality – The degree to which standard interfaces, protocols and bandwidth
are used.
- Completeness – The degree to which full implementation of required function has been achieved.
- Complexity – The degree to which the program is complex.
- Conciseness – The compactness of the program in terms of lines of codes.
- Consistency – The use of uniform design and documentation technique throughout the software
development project.
- Data Commonality – The use of standard data structures and types throughout the program.
McCalls’ Software Quality Factors
http://education.dewsoftoverseas.com/QE/QUickReference/Software%20Enginering/2.6.asp

14. - Error Tolerance – The damage that occurs when the program encounters an error
- Execution Efficiency – The run time performance of a program.
- Expandability – The degree to which architectural, data or procedural data can be extended.
- Generality – The breadth of potential application of program components.
- Hardware Independence – The degree to which the software is decoupled from the hardware on
which it operates.
- Instrumentation – The degree to which the program monitors it’s own operation and identifies
errors that do occur.
- Modularity – The functional independence of program components.
McCalls’ Software Quality Factors (cont’d)
http://education.dewsoftoverseas.com/QE/QUickReference/Software%20Enginering/2.6.asp

15. McCalls’ Software Quality Factors (cont’d)
- Operability – The ease of operation of a program.
- Security – The availability of mechanisms that control or protect programs and data.
- Self-Documentation – The degree to which the source code provides meaningful documentation.
- Simplicity – The degree to which a program can be understood without difficulty.
- Software System Independence – The degree to which the program is independent of nonstandard
programming language features, operating system characteristics, and other environmental
constraints.
- Traceability – The ability to trace a design representation or actual program component back to
requirements.
- Training – The degree to which the software assists in enabling new users to apply the system.
http://education.dewsoftoverseas.com/QE/QUickReference/Software%20Enginering/2.6.asp

16. Metrics by Perspectives

17. Dev’s Perspective

18. LOC
- Lines Of Code
- Commented Lines
- Logic Lines
- New Lines vs Deleted Lines
- Impact (new block vs lots of small changes)
- Bugs Rate

19. Commit Rate

20. Cyclomatic Complexity
https://www.researchgate.net/profile/Vladimir_Riabov/publication/260038796/figure/fig2/AS:297180960182275@1447864870689/Three-methods-of-evaluating-the-cyclomatic-complexity-of-the-graph.png

21. C.R.A.P Index (Change Risk Analysis and Prediction)
https://i.stack.imgur.com/x3XAY.png

22. Code Duplication %
https://i.pinimg.com/originals/8a/39/28/8a3928c74f0e472624a48b741f8a21bc.png

23. Code Coverage %
https://i2.wp.com/blog.mallow-tech.com/wp-content/uploads/2018/05/code_coverage_example.png

24. Broken Tests %
- Marked as Incomplete
- Marked as Skipped
- Risky
- Warnings

25. % New Tests
- Per Sprint
- Per Project

26. Coding Style Violations
https://www.integer-net.com/wp-content/uploads/sites/3/2018/07/grumphp_no.png

27. SonarQube Scores

28. Technical Debt
https://blog.insight.symfony.com/img/screenshots/technical_debt_php.png

29. # 3rd Party Vendors
http://knowledge.wharton.upenn.edu/wp-content/uploads/2016/12/legos.jpg

30. TTFC (Time To First Commit)
● How well-organised is the code base?
● How supportive is the team?
● How much confidence do we have in our tests?
● How mature is the CI pipeline?
● How is the dev environment?

31. # Pending PR

32. Cache Hit Ratio
https://support.stackpath.com/hc/article_attachments/360002767066/Cache_Hit_Ratio.PNG

33. DevOp’s Perspective

34. Broken Builds
https://d3r49iyjzglexf.cloudfront.net/blog/content/insights-1.0-large@2x-f9d4748edfb82ee810b499a39e0671bda4c9e99b849bde044e4ccdee2c5b8e08.gif

35. Build Time
https://www.automatetheplanet.com/wp-content/uploads/2018/04/16-hours-bellatrix-build.png

36. Deploys of Unreviewed PR
https://media-cdn.tripadvisor.com/media/photo-s/0e/21/22/31/the-three-monkeys-cafe.jpg

37. Deploys of Partially Fixed PR
https://media-cdn.tripadvisor.com/media/photo-s/0e/21/22/31/the-three-monkeys-cafe.jpg

38. Deployment Frequency
https://blog.newrelic.com/wp-content/uploads/EOYBlog_Graph_Opt1.jpg

39. Deployment Time

40. Change Fail %
- Rollback
- Manual Patches

41. # Application Errors
https://blog.sentry.io/img/post-images/epsagon/monitored-backend.png

42. Sysadmin’s Perspective

43. Uptime
https://www.pingdom.com/wp-content/themes/pingdom-2018/public/assets/images/uptime-monitoring/uptime-specific-still.png

44. MTTR (Mean Time To Recover)
https://archive.sap.com/image/456241

45. MTBF (Mean Time Between Failures)
https://archive.sap.com/image/456241

46. Size Error Logs
https://preview.ibb.co/eaygbH/Capture_d_e_cran_2018_02_08_11_16_54.png

47. Free Disk Space %
https://i.stack.imgur.com/cRr4H.jpg

48. Disk I/O
https://haydenjames.io/linux-server-performance-disk-io-slowing-application/

49. Bandwidth Saturation %

50. # Infrastructure Alerts
- Disaster
- High
- Average
- All the others

51. Customer Support’s
Perspective

52. Unsolved Issues/Tickets

53. % Returning Problems
https://marketplace-cdn.atlassian.com/files/images/f16e86e7-c5ec-41ae-8c0a-8b0cbec978d4.png

54. FRT (First Response Time)
https://www.geckoboard.com/assets/1-first-response-time-visualizations-customer-support-metric.png

55. Defect Escape Rate
https://image.slidesharecdn.com/webcastpresentation-6-140822160616-phpapp01/95/webcast-presentation-accelerate-continuous-delivery-with-development-testing-and-rational-team-concert-6-638.j
pg

56. Project Manager’s Perspective

57. Lead Time
https://image.slidesharecdn.com/leanleadershipwebinar-part3of3slides-161029141925/95/lean-leadership-part-3-of-3-30-638.jpg

58. Process Time
https://image.slidesharecdn.com/leanleadershipwebinar-part3of3slides-161029141925/95/lean-leadership-part-3-of-3-30-638.jpg

59. Total vs Open
https://www.glostone.com/wp-content/uploads/2016/10/piles-of-paperwork.png

60. Closed Issues/Tickets
- Per Person
- Per Project
- Per Team
- Per Urgency

61. WIP Issues/Tickets
- Per Person
- Per Project
- Per Team
- Per Urgency

62. Estimates Accuracy
https://manifesto.co.uk/wp-content/uploads/2014/08/estimating-in-agile-and-scrum.png

63. Velocity
http://www.softwaretestingstudio.com/wp-content/uploads/2017/05/Initial-Agile-Velocity.png

64. Burndown Variance
https://upload.wikimedia.org/wikipedia/commons/0/05/SampleBurndownChart.png

65. APF (Augmented Pony Factor)
The lowest number of committers whose total contribution
constitutes the majority of the codebase
https://ke4qqq.wordpress.com/2015/02/08/pony-factor-math/

66. HR’s Perspective

67. Tech Test Passing Rate
https://i.kinja-img.com/gawker-media/image/upload/s--EWLoqHzx--/c_scale,fl_progressive,q_80,w_800/18kz3r7mcex5bgif.jpg

68. Team Morale
https://cdn-images-1.medium.com/max/1600/1*3LM47GMbaG9CvSCIHjjPYQ.png

69. Manager’s Perspective

70. PSV (Project Schedule Variance)
http://agilenucleus.com/wp-content/uploads/2015/04/evmchart.png

71. PCV (Project Cost Variance)
http://agilenucleus.com/wp-content/uploads/2015/04/evmchart.png

72. EV (Earned Value)
https://www.pmi.org/kasimage/f58c4893-c369-409a-acf3-227ce7f0074e/p_2.jpg

73. Overall Budget
- Infrastructure (Cloud / Bare Metal)
- Equipment (PC, Devices, Dongles, …)
- Software (Licences, …)
- SaaS
- Personnel
- Consulting
- Outsourcing
- Training (Workshops, Conferences, Books, …)
- Recreational (Pastries, Drinks, Snacks, …)

74. EOL Running Stuff
https://zdnet4.cbsistatic.com/hub/i/2018/10/14/7b07994d-008a-4f9b-82f4-01af045f1646/7f62145966cfb80731a8e9676cd59c13/php-eols.png
https://symfony.com/doc/4.0/_images/release-process.jpg
https://www.dev-metal.com/wp-content/uploads/2014/12/eol-centos.png

75. New Tech Deployed
- ThoughtWorks Radar
- Zalando Radar
- Codemotion Community Radar (WIP)

76. Employee Turnover Rate
https://www.officevibe.com/blog/how-to-calculate-employee-turnover

77. Bus Factor
https://cdn-images-1.medium.com/max/1200/0*A7I4FMXuGkcmJ95A.

78. How Can You Get The Data?

79. - Custom Metrics
- APM (Application Performance Monitoring)
- RUM (Real User Monitoring)
Application Telemetry

80. Internal Tools
- GitHub/GitLab/BitBucket
- Sentry
- SonarQube
- SiteSpeed.io
- StatsD
- Ticket System
- Zabbix/Nagios/Cacti/New Relic/…
- ...

81. What Else?!

82. Graphs

83. Trends

84. Forecasts

85. Aggregates

86. KPIs

87. Pick the Metrics that have the
Most Sense For You

88. Now go, and do your homeworks

89. In-House Tools

90. Scripting

91. Excel

92. InfluxDB & Grafana

93. ELK

94. References
● https://blog.usenotion.com/13-essential-software-development-metrics-to-ensure-quality-219cfc264ed1
● https://www.frontrowagile.com/blog/posts/69-30-metrics-for-agile-software-development-teams
● https://ardalis.com/static-code-analysis-and-quality-metrics
● https://github.com/mre/awesome-static-analysis
● https://www.slideshare.net/JerryTan2/metrics-driven-dev-ops-2017
● https://www.slideshare.net/mattiabattiston/kanban-metrics-in-practice-for-leading-continuous-improvement
● https://github.com/chaoss/metrics
● http://shop.oreilly.com/product/0636920020134.do

95. Thanks!
ANY QUESTIONS?
You can find me here: https://fabiocicerchia.it