Promise08 Wrapup

Slide 1: PROMISE 2008: wrap up Gary D. Boetticher Tim Menzies Tom Ostrand Guenther Ruhe The PROMISE effect: Do more together than separately

Slide 2: The other side of PROMISE  PROMISE is not just an annual meeting,  it is community.  How much contact have you had with your fellow PROMISE-ians during the year?  Are you exploiting the “PROMISE effect”?  Do more together than separately. 2

Slide 3: Case study #1  Cukic & Menzies: new NSF award 2008 to 2011  “Overcoming ceiling effects in defect predictors  Surprisingly good reviews  High praise for the connection to the PROMISE repository  What are the broader impacts of the proposed activity?  “… The work has significant broader impacts and the PROMISE repository that the PIs are a part of provides them a platform for making further contributions to the SE community. …”  Summary Statement  “… The PROMISE repository and the work of the PI represents a significant input to the proposal. …. “  So the repository is now a tool for access funding. 3

Slide 4: Case study #2  For several years, Ostrand, Weyuker, Bell at AT&T held the state of the art in defect localization (predicting the subset of the code with most errors)  2007: At PROMISE’07:  Koru argues that defect distribution is logarithmic (I.e. more errors in the smaller modules)  Sept 2008:  Milton (WVU) tells Menzies that their new defect localizer is failing.  Menzies says “try crawling the suspect modules in the Koru order” (smallest modules first)  Jan’08:  Milton shows the new localizer dramatically out-performs standard methods  Mar’08:  Milton hosted at AT&T for a week by Ostrand  Tests the new localizer inside the firewalls at AT&T 4

Slide 5: Case study #3  TSE 2007, May,  Kitchenham et.al. assess within-company (WC) data vs cross-company (CC) data for effort prediction.  August ‘07 : Menzies (USA) realizes that the PROMISE repository could perform the same study, but for defect prediction.  Crowd sources the PROMISE community for collaborators  Sept ‘07: Bener&Turhan (Turkey) run the experiments  Turkey makes the data, USA runs the statistics  CC results: best PD results ever seen with this data (but very bad PF rates)  Oct ‘07: Submission to TSE by Menzies, Bener, Turhan  CC depreciated for defect prediction  Feb 14: TSE decision: revise & resubmit  May 1: Resubmission with 3 news data sets (Turkish whitegoods)  Paper now much stronger  PROMISE repository now has 3 more data sets  Other Bener, Menzies, Turhan papers: two Defects’08, ASE’08 5

Slide 6: Case study 4,5,6….  Over to you…  Any other stories to add of interactions between PROMISE-ians? 6

Slide 7: The PROMISE effect  Do more together separately.  Look around the room  What ideas are in the air?  What is the value-added of the person sitting next to you?  What are you plans for collaboration over the next 7 months (in time for the next PROMISE cfp)?  And we can use web-tools to foster this community:  Chat rooms, discussion groups, wikis, code library 7

Slide 8: Proposal: a PROMISE chat room  Two minutes chatting can sometimes beat 2 months of reading  ?Meebo.com  Web-based, platform independent 8

Slide 9: Proposal: a PROMISE discussion list  Chatting is a hardly permanent record of a discussion  Good to access what others have said in the past  ?groups.google.com 9

Slide 10: Proposal: a PROMISE wiki  Promisedata.org is read only (except for comments)  Good for workshop admin and data storage  Chat rooms and discussion groups good for discussion “volleys”  Wikis are better for summarizing and synthesizing on-going discussions  write, rewrite access  ?wikispaces.org 10

Slide 11: Proposal: a shared PROMISE code library  Reading is one thing  Running code is another.  Have a shared open-source, code base, complete with make files  Able to reproduce prior results with one command.  ?code.google.com  ?a Debian package 11

Slide 12: Proposal: the PROMISE’09 awards  Source award  Awarded to the individuals or groups that contribute the most new data, with baseline results, to the repository.  Service award  Awarded to the individuals or groups that offer most support to others in the new groups/ chat room/ wiki.  Science award  Awarded to the 2009 paper that best repeats/extends/ interacts with prior results and/or authors of a past PROMISE paper. 12