This document discusses techniques for summarizing various aspects of software development. It begins by describing a software release cycle involving coding, testing, deployment, user feedback, and continuous integration/delivery (CD/CI). Next, it discusses approaches to summarizing source code, test cases, user reviews, and API documentation. Specifically, it describes generating informative summaries of source code classes, test cases, and user reviews to facilitate understanding, and detecting defects in API documentation by analyzing inconsistencies between code and documents. The document emphasizes the importance of summarization in managing software maintenance and evolution in response to changing requirements.

1. Sebastiano Panichella
Institut für Informatik
Universität Zürich
panichella@ifi.uzh.ch
Vienna Software Seminar (2017)
Summarization Techniques
for Code, Change, Testing
and User Feedback

2. Release Cycle
Release
Planning
1
Coding
Activity
Testing
Tasks
Deployment
Phase
Source
Code
Release
App
Reviews
User (or
costumer)
Feedback
2 3
5
6
CD/CI
4
Test
Code

3. Release Cycle
Manual testingAutomated testing
CD/CI
A/B testing
App reviews
Crowdsourcing
Microservices
Team coordination
Code Review Survey
Structured e-mails
Security
Performance Testing
…
……
…
Software documentation
Maintenance activities
Coding activities

4. “In modern software companies it is nowadays, crucial to enact a
software development process able to dynamically react to
market requirements (i.e., users requests), delivering at same
time high quality and reliable software”.
Release Cycle

5. Summarization Approaches

6. Summarization Approaches
“…have the general capability of automatically extracting or abstracting
key content from one or more sources of information thus, determining
the relevant information in the source being summarized and reducing
its content…”
1) indicative summary: it provides a direct link to the required content
relevant sources to users, so that they can read the provided information
more depth.
2) informative summary it has the goal to substitute the origisource
of information, by mainly assembling the relevant content, presenting it
in a new, more concise and structured form.
3) critical summary (or review) it eports or selects the main opinions
or statements related to a specific discussed topic, thus, it brings the
most relevant feedback, both positives and negatives, about a given
subject discussed in the source document.
6

7. Example of informative
summary
MILAN, Italy, April 18. A small airplane crashed into a government
building in heart of Milan, setting the top floors on fire, Italian
police reported. There were no immediate reports on casualties as
rescue workers attempted to clear the area in the city's financial
district. Few details of the crash were available, but news reports
about it immediately set off fears that it might be a terrorist act
akin to the Sept. 11 attacks in the United States. Those fears sent
U.S. stocks tumbling to session lows in late morning trading.
Witnesses reported hearing a loud explosion from the 30-story
office building, which houses the administrative offices of the local
Lombardy region and sits next to the city's central train station.
Italian state television said the crash put a hole in the 25th floor
of the Pirelli building. News reports said smoke poured from the
opening. Police and ambulances rushed to the building in downtown
Milan. No further details were immediately available.

8. MILAN, Italy, April 18. A small airplane crashed into a government
building in heart of Milan, setting the top floors on fire, Italian
police reported. There were no immediate reports on casualties as
rescue workers attempted to clear the area in the city's financial
district. Few details of the crash were available, but news reports
about it immediately set off fears that it might be a terrorist act
akin to the Sept. 11 attacks in the United States. Those fears sent
U.S. stocks tumbling to session lows in late morning trading.
Witnesses reported hearing a loud explosion from the 30-story
office building, which houses the administrative offices of the local
Lombardy region and sits next to the city's central train station.
Italian state television said the crash put a hole in the 25th floor
of the Pirelli building. News reports said smoke poured from the
opening. Police and ambulances rushed to the building in downtown
Milan. No further details were immediately available.
How many victims?
Was it a terrorist act?
What was the target?
What happened?
Says who?
When, where?
Example of informative
summary

9. 35
http://textcompactor.com/
Example of informative
summary

10. 35
http://textcompactor.com/
Example of informative
summary

11. Release Cycle
Maintenance activities
Coding activities
Automated testing
CD/CI
A/B testing
App reviews
Crowdsourcing
Microservices
Team coordination
Code Review Survey
Structured e-mails
Security
Performance Testing
…
…
…
…
?
…
Software documentation
Testing

12. Release Cycle
Maintenance activities
Coding activities
Automated testing
CD/CI
A/B testing
App reviews
Crowdsourcing
Microservices
Team coordination
Code Review Survey
Structured e-mails
Security
Performance Testing
… …
…Software documentation
…
…
Testing

13. Activities in Software Maintenance
Change
documentation
5%
Change
implementation
10%
Change
planning
10%
Change Testing
25%Source code
comprehension
50%
Source: Principles of Software Engineering and
Design, Zelkovits, Shaw, Gannon 1979
Source Code
Summaries: Why?
13

14. Source Code
Summaries: Why?
Prevent Maintenance Cost….
14

15. Understanding Code…
Not So Happy Developers
Happy Developers
Absence of Comments in the Code
again !!
Comments in the Code
again !!
15

16. Understanding Code…
Not So Happy Developers
Happy Developers
Absence of Comments in the Code
again !!
Comments in the Code
again !!
16

17. Source Code
Summaries: How?

18. Questions when Generating
Summaries of Java Classes
■ 1) What information to include in the summaries?
■ 2) How to generate and present the summaries?
18

19. Software Words Usage Model: deriving <actions>, <themes>,
and <secondary arguments> from class, methods, attributes
and variable identifiers
E. Hill et al. Automatically capturing
source code context of NL-queries
for software maintenance and reuse.
ICSE 2009
Summary Generator

20. When Navigating Java Classes…
https://github.com/larsb/atunesplus/blob/master/aTunes/src/main/java/net/sourceforge/atunes/kernel/modules/repository/audio/AudioFile.java
we look at
- Name of the Class
- Attributes
- Methods
- Dependencies between Classes
Source Code
Summaries: How?

21. When Navigating Java Classes…
Source Code
Summaries: How?
■ Generic responsibilities (domain independent)
■ Class stereotypes [Dragan et al., ICSM’10]
■ E.g., data class, entity, controller, boundary,
etc.

22. When Navigating Java Classes…
Source Code
Summaries: How?
■ Generic responsibilities (domain independent)
■ Class stereotypes [Dragan et al., ICSM’10]
■ E.g., data class, entity, controller, boundary,
etc.

23. How to present and
generate the summaries?
Other Code Artefacts can
be Summarised as well:
- Packages
- Classes
- Methods
- etc.http://www.cs.wayne.edu/~severe/jsummarizer/

24. [ Moreno at al. - ICPC 2013 ]
Evaluation of the Summaries

25. Potential Useful Code Descriptions can be
Found in Developers’ Discussions…
source'code'descrip+ons'in'external'
ar+facts.!
..................................................
When call the method IndexSplitter.split(File
destDir, String[] segs) from the Lucene cotrib
directory(contrib/misc/src/java/org/apache/
lucene/index) it creates an index with
segments descriptor file with wrong data.
Namely wrong is the number representing the
name of segment that would be created next in
this index.
..................................................
CLASS:'IndexSplitter! METHOD:'split!
■ Others researchers proposed to detect source code
descriptions from sources external to the source code:
■ Mailing list and Issue tracker
■ StackOverflow Discussions
[ Vassallo at al. - ICPC 2014 ][ Panichella at al. - ICPC 2012 ]

26. Software Changes over the Time…
“…as consequence the original documentation tend to be
incomplete and inconsistent with the source code…”
Insufficient
Information
Source Code
Difficult
to Understand
APIs Documents
Inconsistent withComing back
to the reality...
Inconsistent/
Incomplete
26

27. Source Code
APIs Documents
Inconsistent
API Document Defects are Frequent
API Document JDK-1.8
Class:
InputEvent
Method:
getMaskForButton(int button)
————————————-—————————————-
————————————-—————————————-
https://docs.oracle.com/javase/8/docs/api/java/awt/event/InputEvent.html
“…and tend to be discovered and fixed after long time…”
http://stackoverflow.com/questions/2967303/inconsistency-in-java-util-concurrent-future
27

28. DRONE
DetectoR of dOcumentatioN dEfects
Code
API
Document
Software
Artifacts
AST Parsing
Pre-Process
and POS
Tagging
Defect
Reports
Control Flow-
Based Constraint
Analysis
SMT
Solver
Dependency
Parsing and
Pattern Analysis
Code Constraint
FOL Generating
Doc Constraint
FOL Generating
HeuristicsHeuristics
28

29. DRONE
DetectoR of dOcumentatioN dEfects
Code
API
Document
Software
Artifacts
AST Parsing
Pre-Process
and POS
Tagging
Defect
Reports
Control Flow-
Based Constraint
Analysis
SMT
Solver
Dependency
Parsing and
Pattern Analysis
Code Constraint
FOL Generating
Doc Constraint
FOL Generating
HeuristicsHeuristics
29

30. DRONE
DetectoR of dOcumentatioN dEfects
Code
API
Document
Software
Artifacts
AST Parsing
Pre-Process
and POS
Tagging
Defect
Reports
Control Flow-
Based Constraint
Analysis
SMT
Solver
Dependency
Parsing and
Pattern Analysis
Code Constraint
FOL Generating
Doc Constraint
FOL Generating
HeuristicsHeuristics
30

31. DRONE
DetectoR of dOcumentatioN dEfects
Code
API
Document
Software
Artifacts
AST Parsing
Pre-Process
and POS
Tagging
Defect
Reports
Control Flow-
Based Constraint
Analysis
SMT
Solver
Dependency
Parsing and
Pattern Analysis
Code Constraint
FOL Generating
Doc Constraint
FOL Generating
HeuristicsHeuristics
31

32. Code
API
Document
Software
Artifacts
AST Parsing
Pre-Process
and POS
Tagging
Defect
Reports
Control Flow-
Based Constraint
Analysis
SMT
Solver
Dependency
Parsing and
Pattern Analysis
Code Constraint
FOL Generating
Doc Constraint
FOL Generating
HeuristicsHeuristics
DRONE
Documentation Code
With DRONE we analyzed
over 1 million of LOC and more than
30,000 Javadoc documents
belonging to 8 java libraries
detecting around 2000 of
API documentation defects.
with high precision (values between
0.58 - 0.83) and an high recall (values >
0.81) results.
“Analyzing APIs Documentation and Code to Detect Directive Defects”. ICSE 2017
Evaluation of the Summaries
32

33. Release Cycle
Maintenance activities
Coding activities
Automated testing
CD/CI
A/B testing
App reviews
Crowdsourcing
Microservices
Team coordination
Code Review Survey
Structured e-mails
Security
Performance Testing
…
…Software documentation…
Testing

34. Release Cycle
Maintenance activities
Coding activities
CD/CI
A/B testing
App reviews
Crowdsourcing
Microservices
Team coordination
Code Review Survey
Structured e-mails
Security
Performance Testing
…
…… Software documentation
Automated testing Testing

35. Release Cycle
Testing
CD/CI
A/B testing
App reviews
Crowdsourcing
Microservices
Team coordination
Code Review Survey
Structured e-mails
Security
Performance Testing
…
…… Software documentation
Automated testing
Maintenance activities
Coding activities

36. Test Cases Summarization

37. Example of Test Case
Generated by Evosuite
Test Case Automatically
Generated by Evosuite
(for the class apache.commons.Option.Java)
}
37

38. Example of Test Case
Generated by Evosuite
Test Case Automatically
Generated by Evosuite
(for the class apache.commons.Option.Java)
Not Meaningful
Names for Test Methods
It is difficult to tell, without
reading the contents of the
target class, what is the
behavior under test.
}
38

39. Test Case Automatically
Generated by Evosuite
(for the class apache.commons.Option.Java)
Example of Test Case
Generated by Evosuite
39

40. Test Case Automatically
Generated by Evosuite
(for the class apache.commons.Option.Java)
Our Solution: Automatically Generate
Summaries of Test Cases
40

41. Results
30 Developers:
- 22 Researchers
- 8 Professional Developers
41

42. Release Cycle
Testing
Maintenance Activities
Coding Activities
CD/CI
A/B testing
App reviews
Crowdsourcing
Microservices
Team coordination
Code Review Survey
Structured e-mails
Security
Performance Testing
… …
…
…
… Software documentation
Automated testing

43. Release Cycle
Testing
Maintenance activities
Coding activities
CD/CI
A/B testing
App reviews
Crowdsourcing
Microservices
Team coordination
Code Review Survey
Structured e-mails
Security
Performance Testing
… …
…
…
… Software documentation
Automated testing

44. Maintenance of Mobile Applications
“About one third of app reviews
contain useful information for developers”
Pagano et. al. RE2013 44

45. Feature Requests Bug Reports
The Problem
45

46. SURF (Summarizer of User Reviews Feedback)
Summaries of User Reviews
What Would Users Change in My App? Summarizing App Reviews for Recommending Software Changes.
24th ACM SIGSOFT International Symposium on the Foundations of Software Engineering (FSE) - 2016.
SURF: Summarizer of User Reviews Feedback.
Proceedings of the 39th IEEE International Conference on Software Engineering (ICSE) - 2017
46

47. Summary Generation
http://www.ifi.uzh.ch/en/seal/people/panichella/tools/SURFTool.html
SURF: Summarizer of User Reviews Feedback.
Proceedings of the 39th IEEE International Conference on Software Engineering (ICSE) - 2017
47

48. Case Study
Involving 23 Developers
2622
Reviews
Of
12
Apps
48

49. Results
V.S.!
2) SURF helps to prevent more than half
of the time required for analyzing users
feedback and planning software changes.
3) 92% of manually extracted
feedback appears also in the
automatic generated summaries.
4) Summaries generated by SURF
are reasonably correct, adequate,
concise, and expressive.
49

50. TestingAutomated testing
CD/CI
A/B testing
Crowdsourcing
Microservices
Team coordination
Survey
Structured e-mails
Security
Performance Testing
… …
…
…
…
…
Software documentation
Code Review
Coding Activities
Maintenance Activities
App reviews
Conclusion

51. G. Grano, A. Ciurumelea, S. Panichella, F. Palomba, H. Gall
SANER - 2018
Giovanni Grano

52. Thanks!