The document discusses the importance of software analytics in solving problems through automated, data-centric, and reproducible methods. It emphasizes the need for clarity in decision-making and continuous learning in software engineering, while highlighting various tools and techniques for data retrieval, analysis, and visualization. Several examples are provided to illustrate the application of these concepts in real-world scenarios.

1. Software Analytics
for Pragmatists
Solving problems –
automated, data-centric and reproducible
Markus Harrer
software analytics clean code

2. code

3. “Without data you're
just another person
with an opinion.“
W. Edwards Deming

4. Motivation Software Analytics
“In software engineering
there is much we are seeing,
but little we are learning.”
Tim Menzies

5. State of the Art

6. State of the Art

7. Questions from daily work

8. Motivation
Why?
•Make problems visible
+ Improve clarity and understanding
•Drive decisions
+ Raise money
+ Raise more money for further analysis
•Support Continuous Learning
+ Master challenges
+ Thrive for improvement steadily

9. Claim
Solving problems –
automated,
data-centric and
reproducible.

10. Claim
Solving problems –
automated,
data-centric and
reproducible.

11. Automation
•Data retrieval
•Analysis
•Visualization
DevOps style

12. Claim
Solving problems –
automated,
data-centric and
reproducible.

13. Types of software artifacts and meta data
chronologicalcommunity
runtimestatic

14. Data is dirty
Time for cleaning up data
up to 80%

15. Claim
Solving problems –
automated,
data-centric and
reproducible.

16. Data + Automation + Traceability
Replication

17. Code Book
Example:
Analyzing performance bottlenecks

18. Claim
Solving problems –
automated,
data-centric and
reproducible.

19. Pipeline
Data Mining
• NumPy
• scikit-learn
• SciPy
Visualization
• matplotlib
• plot.ly
• Bokeh
• python-pptx
...

20. Pipeline
XML/Graph
Tabellen
matplotlib
Pandas,
...
Pandas
jQAssistant,
Neo4j
Text
xlsx
E
pptx
P
Python Jupyter
Input
Pre-
processing
Analysis
Output
D3

21. Pipeline
ZIP
GZ
*.class
JAR, WAR, EAR
MANIFEST.MF
*.properties
XSD
YAML
XML
application.xml
web.xml
beans.xml
JaCoCo
FindBugs
CheckStyle
pom.xml
surefire-reports.xml RDBMS Schema
M2 Repository
DBCSV
Excel
BigQuery
Inputs
HDFStore
Web
JSON
Git
Pandas
jQAssistant

22. Examples
Demos
•Performance Bottlenecks
•Build Breaker Analysis
•Mining Knowledge Islands
•Git Log Analysis

23. Experience
Learning by Doing!

24. What‘s the
value of the information
related to the
effort for the analysis
of the information?
Always ask the question

25. References
Leek, Jeff: The Elements of Data Analytic Style.
LeanPub, 2015.
McKinney, Wes: Python For Data Analysis, O’Reilly,
2012.
Mens, Tom; Serebrenik; Cleve, Anthony:Evolving
Software Systems. Springer, 2014.
Mens, Tom; Demeyer, Serge: Software Evolution.
Springer, 2008.
Shull, Forrest; Singer, Janice; Sjøberg, Dag I.K.: Guide
to Advanced Empirical Software Engineering. Springer,
2008.
Tornhill, Adam:Your Code As a Crime Scene.
Pragmatic Programmers, 2015.

26. Your Questions