ESEM 2014 conference

1. Tracing Back the History of Commits
in Low-tech Reviewing Environments
Yujuan Jiang, Bram Adams, Daniel German and Foutse Khomh
1

3. 3
linux-usb
linux-scsi
lkml
subsystem
maintainer1
subsystem
maintainer1
Reviewing: patch Integration: commit
maintainer Linus Torvalds
Email-based Reviewing Environment
contributor
contributor
contributor

4. SS MS
MM

5. Data Collection
Emails from mailing listsCommits from Git Repo
5
Linking
CCFinder
(token-level)
+/- line-based
(line-level)
Checksum-based
(Chunk-level)

6. Research Questions
RQ1: Can commits
be linked accurately
to emails containing
the corresponding
patch version?
RQ2: Can emails
containing different
patch versions be
linked accurately to
each other?
RQ3: What are the
characteristics of the
reviewing history in a
low-tech reviewing
environment?
6

7. Evaluation: Precision
7
384 samples of a technique

8. A BRelative
Recall A
= 4/6=67%
Relative
Recall B
= 3/6=50%
8
Evaluation: Relative Recall

9. Evaluation: Ground Truth
Fix one security bug [v2] Fix one security bug [v3] Fix one security bug

10. Case Study Result
RQ1: Can commits
be linked accurately
to emails containing
the corresponding
patch version?
RQ2: Can emails
containing different
patch versions be
linked accurately to
each other?
RQ3: What are the
characteristics of the
reviewing history in a
low-tech reviewing
environment?
10

11. Plus/Minus line technique has highest
F-measure to link email patch to commit
Table1: statistics of email-commit links
+/- line result has
highest relative
recall
checksum result has
highest precision
+/- result has
highest F-measure

12. Plus/Minus line technique has highest
F-measure to link email patch to commit
take up
more than 85%

13. Research Questions
RQ1:
be linked accurately
to emails containing
the corresponding
patch version?
RQ2: Can emails
containing different
patch versions be
linked accurately to
each other?
RQ3: What are the
characteristics of the
reviewing history in a
low-tech reviewing
environment?
13

14. Checksum technique has highest
F-measure to link email to email
Table2: statistics of email-email links
checksum result has
highest precision
+/- line result has
highest relative recall
+/- line result has
highest F-measure

15. Checksum technique has highest
F-measure to link email to email
checksum&+/-
takes up around
95%

16. Research Questions
RQ1:
be linked accurately
to emails containing
the corresponding
patch version?
RQ2:
containing different
patch versions be
linked accurately to
each other?
RQ3: What are the
characteristics of the
reviewing history in a
low-tech reviewing
environment?
16

17. 17
25% of the MM patches has
“hidden” reviewing history
of more than four weeks.
Larger and impact more files
A new thread is started
if too much time has passed
More bug-prone Higher acceptance rate