• Share
  • Email
  • Embed
  • Like
  • Save
  • Private Content
Expert Recommendation with Usage Expertise
 

Expert Recommendation with Usage Expertise

on

  • 2,532 views

Presented at ICSM 2009

Presented at ICSM 2009

Statistics

Views

Total Views
2,532
Views on SlideShare
2,532
Embed Views
0

Actions

Likes
0
Downloads
0
Comments
0

0 Embeds 0

No embeds

Accessibility

Categories

Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

    Expert Recommendation with Usage Expertise Expert Recommendation with Usage Expertise Presentation Transcript

    • Expert Recommendation with Usage Expertise David Ma David Schuler University of Calgary Saarland University Tom Zimmermann Jonathan Sillito Microsoft Research University of Calgary 1
    • Stuck... AutoBuildJob(Workspace workspace) { super(Messages.events_building_0); setRule(workspace.getRoot()); setPriority(BUILD); isAutoBuilding = workspace.isAutoBuilding(); preferences.addPropertyChangeListener(this); } 2
    • Who knows how this code works? AutoBuildJob(Workspace workspace) { super(Messages.events_building_0); setRule(workspace.getRoot()); setPriority(BUILD); isAutoBuilding = workspace.isAutoBuilding(); preferences.addPropertyChangeListener(this); } 3
    • Who knows how this code works? Who has modified this code before? 4
    • AutoBuildJob(Workspace workspace) { super(Messages.events_building_0); setRule(workspace.getRoot()); setPriority(BUILD); isAutoBuilding = workspace.isAutoBuilding(); preferences.addPropertyChangeListener(this); } 5
    • AutoBuildJob(Workspace workspace) { super(Messages.events_building_0); setRule(workspace.getRoot()); setPriority(BUILD); isAutoBuilding = workspace.isAutoBuilding(); preferences.addPropertyChangeListener(this); } Anna changes AutoBuildJob. She knows how this method works. 6
    • Implementation Expertise AutoBuildJob(Workspace workspace) { super(Messages.events_building_0); setRule(workspace.getRoot()); setPriority(BUILD); isAutoBuilding = workspace.isAutoBuilding(); preferences.addPropertyChangeListener(this); } Anna changes AutoBuildJob. She knows how this method works. 7
    • Implementation Expertise Projects unlikely to share code This concept can’t transfer expertise across projects 8
    • Developers moving between multiple projects or that are new to projects Implementation Expertise New projects don’t have data Some projects don’t have enough data or is of poor quality 9
    • 10
    • Usage Expertise AutoBuildJob(Workspace workspace) { super(Messages.events_building_0); setRule(workspace.getRoot()); setPriority(BUILD); isAutoBuilding = workspace.isAutoBuilding(); preferences.addPropertyChangeListener(this); } Matt adds a call to setPriority(). Matt knows how to use setPriority() without knowing how it was coded. 11
    • Usage Expertise and Context AutoBuildJob(Workspace workspace) { super(Messages.events_building_0); setRule(workspace.getRoot()); setPriority(BUILD); isAutoBuilding = workspace.isAutoBuilding(); preferences.addPropertyChangeListener(this); } Matt adds a call to setPriority(). Matt knows how to use setPriority() and the surrounding method calls. 12
    • Why Usage Expertise? Projects unlikely to share code Projects likely to share calls to libraries 13
    • How Do We Make Recommendations? 14
    • Expertise Profiles Database of Profiles Mine developer history for Bob’s Profile Changed Methods and Added Method Calls Dan’s Profile Alice’s Profile 15
    • Looking for Experts Who has Database of Profiles Implementation Expertise? AutoBuildJob(Workspace workspace) { Bob’s Profile super(Messages.events_building_0); setRule(workspace.getRoot()); setPriority(BUILD); isAutoBuilding = workspace.isAutoBuilding(); preferences.addPropertyChangeListener(this); } Dan’s Profile Alice’s Profile Who has Usage Expertise? 16
    • Ranking Implementation Expertise (How previous research has done it) Change Change Recency Frequency 17
    • Ranking Usage Expertise Frequency of Breadth of Usage Usage Frequency of Breadth of Usage Relative Usage Relative to Everyone Else to Everyone Else 18
    • Research Questions Can Usage Expertise correctly 1 recommend experts within projects as often as Implementation Expertise? Can Usage Expertise + Context correctly 2 recommend experts within projects as often as Implementation Expertise? Is it possible to recommend developers 3 across projects? 19
    • Experiment 1 of 2 Recommending Within Projects accuracy of implementation-based recommendations vs. accuracy usage-based recommendations 20
    • Training Database of Profiles Bob’s Profile CVS history Dan’s Profile Alice’s Profile 21
    • Testing Database of Changeset Ranked by Joe Profiles impl. expertise Matt Changed Bob (expert) + Methods Ranked by Added usage expertise Method Kim Calls Bob (expert) Joe Evaluation Did we recommend the actual expert? 22
    • Experiment 2 of 2 Recommending Across Projects Determining if usage-based recommendations are possible across projects 23
    • Training Database of Profiles Eclipse CVS history Bob’s Profile Jack’s Profile AspectJ CVS history Dan’s Profile Alice’s Profile Kate’s Profile 24
    • Testing Database of Changeset Profiles Changed Ranked by usage expertise + Methods Alice (Eclipse) Added Jake (Eclipse) Method Kim (AspectJ) Calls Evaluation Did we recommend devs. from other projects? 25
    • Results (Within Projects) Impl-based recommendations are correct ~80% of time time while usage based are correct ~50% of the time. 26
    • Can Usage Expertise correctly 1 recommend experts within projects as often as Implementation Expertise? No 27
    • Results (Within Projects) Usage Expertise + Context correctly recommends experts just as often as Implementation Expertise 28
    • Can Usage Expertise + Context correctly 2 recommend experts within projects as often as Implementation Expertise? YES 29
    • Results (Across Projects) Our usage expertise heuristics recommend developers 25-75% of the time from other projects. 30
    • Is it possible to recommend developers 3 across projects? YES 31
    • Is it possible to recommend developers 3 across projects? YES BUT 32
    • Open Questions How precise are recommendations made across projects? Can we recommend for projects with no or little history? What if we combine both implementation and usage expertise? 33
    • Research Questions Can Usage Expertise correctly recommend experts within projects as often as Implementation Expertise? No Can Usage Expertise + Context correctly recommend experts within projects as often as Implementation Expertise? YES Is it possible to recommend developers across projects? YES 34