Context Is King: The Developer Perspective on the Usage of Static Analysis Tools

1. 1
Context Is King:
The Developer Perspective on the Usage of Static Analysis Tools.
Carmine Vassallo, Sebastiano Panichella, Fabio Palomba,
Sebastian Proksch, Andy Zaidman, and Harald Gall. @ccvassallo

2. 2
Development Context Is King:
The Developer Perspective on the Usage of Static Analysis Tools.
Carmine Vassallo, Sebastiano Panichella, Fabio Palomba,
Sebastian Proksch, Andy Zaidman, and Harald Gall. @ccvassallo

3. 3
Development Context Is King:
The Developer Perspective on the Usage of Static Analysis Tools.
Carmine Vassallo, Sebastiano Panichella, Fabio Palomba,
Sebastian Proksch, Andy Zaidman, and Harald Gall. @ccvassallo

4. 4
Motivation

5. 5
ASAT
(A) Automatic
(S) Static
(A) Analysis
(T) Tool

6. 6
ASATs detect so+ware defects faster and cheaper than human
inspec6on and tes6ng would do (Johnson et al., ICSE 2013).
ASATs are common, but not ubiquitous (Beller et al., SANER 2016)

7. 7
Barriers when using ASATs
Lack of effec6vely
implemented quick fixes
Johnson et al.,
“Why don’t software developers use
Static Analysis Tools to Find Bugs?”
ICSE 2013
High rate of false posi6ve
warnings
Low understandability of the
warnings

8. 8
Usage of ASATs in one context
Panichella et al.,
“Would static analysis tools help
developers with code reviews?”
SANER 2015
Zampetti et al.,
“How open source projects use static
code analysis tools in continuous
integration”
MSR 2017
Build failures caused by ASATs are mainly
due to coding standard viola.ons
Developers use ASATs mainly for checking
coding structure
Code Review
Continuous
Integration

9. ASATs configured differently in
different contexts.
9

10. TO DO List
10
Development Contexts where ASATs are used
Usage of ASATs in different contexts

11. First Study:
Development Contexts
11

12. First Study: Research Questions
• RQ1: In which development contexts do
developers use ASATs?
• RQ2: How do developers configure ASATs in
different development contexts?
12

13. The Questionnaire
13
19 questions, 2 main topics:
• Adoption of ASATs
• Configuration of ASATs
43 (69% industrial and 31%
open-source) participants.

14. Usage of ASATs
14
Frequency
Multiple times per day
Daily
Weekly
Monthly
% Respondents
0 10 20 30 40
12
19
31
38
ASATs are integrated with the regular development

15. 15
Where ASATs are used

16. Where ASATs are used
16
30% 33% 37%
% Respondents
Local Development Code Review Continuous Integration

17. When ASATs are configured
17
Frequency
Kick-off
Monthly
Never
Weekly
% Respondents
0 15 30 45 60
7
20
22
51
The majority of developers configure ASATs only once.

18. 18
How ASATs are configured
of our respondents use the
same configuration in
different contexts.%75

19. How ASATs are configured
19
Local Development Code Review Continuous Integration

20. Second Study:
ASATs usage in different contexts
20

21. Extended questionnaire & Interviews
21
Context-Based Usage
25 participants
11 professional
developers
• 6 companies
Semi-structured
interviews

22. Second Study: Research Question
• RQ3 Do developers pay attention to the same
warnings in different development contexts?
22

23. Warnings in different contexts
23
Local Development Code Review Continuous Integration
Developers pay attention to different warnings depending on the context.
Code Structure
Logic
Error Handling
Style Convention
Redundancies
Naming Conventions
Error Handling
Logic
Style Convention
1st
2nd
3rd
1st
2nd
3rd
1st
2nd
3rd

24. Other factors while selecting warnings
24
Factors
Severity of the Warnings
Policies of the Development Team
Application Type
Team Composition
None of the above
Tool Reputation
% Respondents
0 15 30 45 60
0
6.1
6.1
12.1
24.2
51.5
2.4
2.4
9.9
19.5
31.7
34.1
2.3
7
11.6
18.6
27.9
32.6
Continuous Integration Code Review Local Development
Blocker, Cri>cal, Major, etc.
“Team leader decides to adopt a strict
policy regarding naming conven6ons.”
“Short-term applica6ons don’t need to
follow strict rules.”

25. 25
Findings

26. How developers configure ASATs
26
Local Development Code Review Continuous Integration

27. How developers perceive ASATs
27
Local Development Code Review Continuous Integration

28. 28
Biased Percep6on
Towards Context-Awareness
Holis6c Analysis of the
Developers’ Behaviour

29. 29
Context Is King:
The Developer Perspective on the Usage of Static Analysis
Tools.
Carmine Vassallo, Sebastiano Panichella, Fabio Palomba,
Sebastian Proksch, Andy Zaidman, and Harald Gall.
@ccvassallo
vassallo@ifi.uzh.ch
X
Usage of ASATs in one context
Panichella et al.,
“Would static analysis tools help
developers with code reviews?”
SANER 2015
Zampetti et al.,
“How open source projects use static
code analysis tools in continuous
integration”
MSR 2017
Code Review
Continuous
Integration
How ASATs are configured
X
Local Development Code Review Continuous Integration
How developers perceive ASATs
X
Local Development Code Review Continuous Integration