Tao XieNorth Carolina State UniversityRaleigh, NC, USARAISE 2013
IBMs Deep Blue defeated chess championGarry Kasparov in 1997IBMWatson defeated top human Jeopardy!players in 2011
Google’s driverless carMicrosofts instant voice translation toolIBMWatson as Jeopardy! player
"Completely AutomatedPublic Turing test to tellComputers and HumansApart"
Movie: Minority ReportCNN NewsiPad
…
 Machine is better at task set A Mechanical, tedious, repetitive tasks, … Ex. solving constraints along a long path Hu...
Malaysia Airlines Flight 124@2005Lisanne Bainbridge, "Ironies of Automation”,Automatica 1983 .Ironies of Automation“Even h...
Malaysia Airlines Flight 124@2005Lisanne Bainbridge, "Ironies of Automation”,Automatica 1983 .Ironies of Automation“The in...
 Don’t forget human intelligence Using your tools as end-to-end solutions Helping your tools Don’t forget cooperations...
 Don’t forget human intelligence Using your tools as end-to-end solutions Helping your tools Don’t forget cooperations...
14“During the past 21 years, over 75 papers and 9Ph.D. theses have been published on pointeranalysis.Given the tones of wo...
15Section 4.3 Designing an Analysis for a Client’s Needs“Barbara Ryder expands on this topic: “…We can all writean unbound...
17Zhenmin Li, Shan Lu, Suvda Myagmar, andYuanyuan Zhou. CP-Miner: a tool forfinding copy-paste and related bugs inoperatin...
18Available inVisual Studio 2012Searching similar snippets forfixing bug onceFinding refactoringopportunityYingnong Dang, ...
19XIAO enables code clone analysis withHigh scalability, High compatibilityHigh tunability: what you tune is what you getH...
 50 years of automated debugging research N papers  only 5 evaluated with actual programmers“”Chris Parnin and Alessand...
 Academia Tend to leave human out of loop (involving human makesevaluations difficult to conduct or write) Tend not to ...
 Goal: to identify the future directions in researchin formal methods and its transition to industrialpractice. The work...
 “Lack of education amongst practitioners” “Education of students in logic and design forverification” “Expertise requi...
 “Not integrated with standard development flows(testing)” “Too many false positives and no ranking of errors” “General...
 “The necessity of detailed specifications andcomplex interaction with tools, which is very costlyand discouraging for in...
2010 Dagstuhl Seminar 10111Practical Software Testing: Tool Automation and Human Factorshttp://www.dagstuhl.de/programm/ka...
2010 Dagstuhl Seminar 10111Practical Software Testing: Tool Automation and Human FactorsHuman Factorshttp://www.dagstuhl.d...
Andy Ko and Brad Myers. Debugging Reinvented: Asking and Answering Why and Why NotQuestions about Program Behavior. In Pro...
 Don’t forget human intelligence Using your tools as end-to-end solutions Helping your tools Don’t forget cooperations...
 Motivation Architecture recovery is challenging (abstraction gap) Human typically has high-level view in mind Repeat...
Running Symbolic PathFinder ...…====================================================== resultsno errors detected==========...
 object-creation problems (OCP) - 65% external-method call problems (EMCP) – 27%Total block coverage achieved is 50%, lo...
33 A graph example fromQuickGraph library Includes two classesGraphDFSAlgorithm GraphAddVertexAddEdge: requiresboth ver...
34 Test target: Cover truebranch (B4) of Line 24 Desired objectstate: graph shouldinclude at least oneedge Target seque...
 object-creation problems (OCP) - 65% external-method call problems (EMCP) – 27%Total block coverage achieved is 50%, lo...
36
Total block coverage achieved is 50%, lowest coverage 16%.37 Ex: Dynamic Symbolic Execution (DSE) /ConcolicTesting[Godefr...
2010 Dagstuhl Seminar 10111Practical SoftwareTesting: Tool Automation and Human Factors
 Tackling object-creation problems Seeker [OOSPLA 11] , MSeqGen [ESEC/FSE 09]Covana [ICSE 2011], OCAT [ISSTA 10]Evacon [...
40 Test target: Cover truebranch (B4) of Line 24 Desired objectstate: graph shouldinclude at least oneedge Target seque...
Tackle object-creation problems with Factory Methods41
Tackle external-method call problems with Mock Methods orMethod InstrumentationMocking System.IO.File.ReadAllText42
 Human-Assisted Computing Driver: tool Helper: human Ex. Covana [Xiao et al. ICSE 2011] Human-Centric Computing Dri...
 Motivation Tools are often not powerful enough Human is good at some aspects that tools are not What difficulties doe...
 Human-Assisted Computing Driver: tool Helper: human Ex. Covana [Xiao et al. ICSE 2011] Human-Centric Computing Dri...
1,230,309 clicked Ask Pex!www.pexforfun.com46NikolaiTillmann, Jonathan De Halleux,Tao Xie, Sumit Gulwani and Judith Bishop...
Secret Implementationclass Secret {public static int Puzzle(int x) {if (x <= 0) return 1;return x * Puzzle(x-1);}}Player I...
 Coding duels at http://www.pexforfun.com/ Brain exercising/learning while having fun Fun: iterative, adaptive/personal...
http://pexforfun.com/gradsoftengObserved Benefits• Automatic Grading• Real-time Feedback (for Both Students andTeachers)• ...
“It really got me *excited*.The part that got me most isabout spreading interest in teaching CS: I do think that it’sREALL...
52Internetclass Secret {public static int Puzzle(int x) {if (x <= 0) return 1;return x * Puzzle(x-1); } } Everyone cancon...
InternetPuzzle Games Made fromDifficult Constraints or Object-Creation ProblemsSupported by MSR SEIFAwardNing Chen and Sun...
http://www.cs.washington.edu/verigames/
55Pattern MatchingBug updateProblematicPattern RepositoryBug DatabaseTrace analysisBugfilingStackMine [Han et al. ICSE 12]...
“We believe that the MSRA tool is highly valuable andmuch more efficient for mass trace (100+ traces) analysis.For 1000 tr...
 Don’t forget human intelligence Using your tools as end-to-end solutions Helping your tools Don’t forget cooperations...
 Human-Assisted Computing Human-Centric Computing Human-Human Cooperation
 Wonderful current/former students@NCSU ASE Collaborators, especially those from MicrosoftResearch Redmond/Asia, Peking ...
Questions ?https://sites.google.com/site/asergrp/
Synergy of Human and Artificial Intelligence in Software Engineering
Synergy of Human and Artificial Intelligence in Software Engineering
Synergy of Human and Artificial Intelligence in Software Engineering
Synergy of Human and Artificial Intelligence in Software Engineering
Upcoming SlideShare
Loading in...5
×

Synergy of Human and Artificial Intelligence in Software Engineering

3,927

Published on

Keynote Talk by Tao Xie at International NSF sponsored Workshop on Realizing Artificial Intelligence Synergies in Software Engineering (RAISE 2013) http://promisedata.org/raise/2013/

Published in: Technology, Spiritual
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
3,927
On Slideshare
0
From Embeds
0
Number of Embeds
2
Actions
Shares
0
Downloads
37
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide
  • http://techyyouth.com/google-driverless-car/http://www.lackuna.com/2012/03/15/microsoft-develops-translation-tool-that-mimics-users-own-voice/
  • “Even highly automated systems, such as electric power networks, need human beings... one can draw the paradoxical conclusion that automated systems still are man-machine systems, for which both technical and human factors are important.’http://spectrum.ieee.org/computing/software/automated-to-deathhttp://spectrum.ieee.org/riskfactor/computing/it/are-we-automating-ourselves-into-a-corner
  • “Even highly automated systems, such as electric power networks, need human beings... one can draw the paradoxical conclusion that automated systems still are man-machine systems, for which both technical and human factors are important.’http://spectrum.ieee.org/computing/software/automated-to-deathhttp://spectrum.ieee.org/riskfactor/computing/it/are-we-automating-ourselves-into-a-corner
  • To study what problems
  • To study what problems
  • To study what problems
  • http://www.google.com.hk/url?sa=t&amp;source=web&amp;cd=1&amp;ved=0CBwQFjAA&amp;url=https%3A%2F%2Fwww.iam.unibe.ch%2Fscg%2Fsvn_repos%2FLectures%2FArchive%2FOORPT-W05%2F11ArchitectureRecovery.ppt&amp;ei=PnLPTf7CBoGGuQPu0cCTCg&amp;usg=AFQjCNF7yXlmr5r5rUd8NEf2YYltA8-erA
  • To study what problems
  • To study what problems
  • To study what problems
  • To study what problems
  • To study what problems
  • To study what problems
  • http://developers.de/blogs/damir_dobric/archive/2009/04/13/using-of-factory-in-pex.aspx
  • http://geekswithblogs.net/thomasweller/archive/2010/04/28/mocking-the-unmockable-using-microsoft-moles-with-gallio.aspx
  • http://www.mysvc.it/myapps/constraints/
  • http://www.mysvc.it/myapps/constraints/
  • Synergy of Human and Artificial Intelligence in Software Engineering

    1. 1. Tao XieNorth Carolina State UniversityRaleigh, NC, USARAISE 2013
    2. 2. IBMs Deep Blue defeated chess championGarry Kasparov in 1997IBMWatson defeated top human Jeopardy!players in 2011
    3. 3. Google’s driverless carMicrosofts instant voice translation toolIBMWatson as Jeopardy! player
    4. 4. "Completely AutomatedPublic Turing test to tellComputers and HumansApart"
    5. 5. Movie: Minority ReportCNN NewsiPad
    6. 6.
    7. 7.  Machine is better at task set A Mechanical, tedious, repetitive tasks, … Ex. solving constraints along a long path Human is better at task set B Intelligence, human intent, abstraction, domainknowledge, … Ex. local reasoning after a loop, recognizing namingsemantics= A U B 8
    8. 8. Malaysia Airlines Flight 124@2005Lisanne Bainbridge, "Ironies of Automation”,Automatica 1983 .Ironies of Automation“Even highly automated systems, such aselectric power networks, need human beings...one can draw the paradoxical conclusion thatautomated systems still are man-machinesystems, for which both technical and humanfactors are important.”“As the plane passed 39 000 feet, the stall and overspeedwarning indicators came on simultaneously—somethingthat’s supposed to be impossible, and a situation the crewis not trained to handle.” IEEE Spectrum 2009
    9. 9. Malaysia Airlines Flight 124@2005Lisanne Bainbridge, "Ironies of Automation”,Automatica 1983 .Ironies of Automation“The increased interest in human factors amongengineers reflects the irony that the moreadvanced a control system is, so the morecrucial may be the contribution of the humanoperator.”
    10. 10.  Don’t forget human intelligence Using your tools as end-to-end solutions Helping your tools Don’t forget cooperations of human and toolintelligence; human and human intelligence Human can help your tools too Human and human could work together to help yourtools, e.g., crowdsourcing11
    11. 11.  Don’t forget human intelligence Using your tools as end-to-end solutions Helping your tools Don’t forget cooperations of human and toolintelligence; human and human intelligence Human can help your tools too Human and human could work together to help yourtools, e.g., crowdsourcing12
    12. 12. 14“During the past 21 years, over 75 papers and 9Ph.D. theses have been published on pointeranalysis.Given the tones of work on this topicone may wonder, “Havent we solved thisproblem yet?With input from manyresearchers in the field, this paper describesissues related to pointer analysis and remainingopen problems.”Michael Hind. Pointer analysis: havent we solved this problem yet?. In Proc.ACMSIGPLAN-SIGSOFT Workshop on Program Analysis for SoftwareTools andEngineering (PASTE 2001)
    13. 13. 15Section 4.3 Designing an Analysis for a Client’s Needs“Barbara Ryder expands on this topic: “…We can all writean unbounded number of papers that compare differentpointer analysis approximations in the abstract.However, this does not accomplish the key goal, which isto design and engineer pointer analyses that are usefulfor solving real software problems for realisticprograms.”
    14. 14. 17Zhenmin Li, Shan Lu, Suvda Myagmar, andYuanyuan Zhou. CP-Miner: a tool forfinding copy-paste and related bugs inoperating system code. In Proc. OSDI2004.MSRAXIAOYingnong Dang, Dongmei Zhang, Song Ge,Chengyun Chu,Yingjun Qiu, andTao Xie.XIAO:Tuning Code Clones at Hands ofEngineers in Practice. In Proc. ACSAC2012,MSR 2011 Keynote byYY Zhou: ConnectingTechnology withReal-world Problems – From Copy-paste Detection toDetecting Known BugsHuman Intelligence toDetermine What areSerious Bugs
    15. 15. 18Available inVisual Studio 2012Searching similar snippets forfixing bug onceFinding refactoringopportunityYingnong Dang, Dongmei Zhang, SongGe,YingjunQiu, andTao Xie. XIAO:Tuning Code Clones at Hands of Engineers inPractice. In Proc. AnnualComputer Security ApplicationsConference (ACSAC 2012)XIAO Code Clone Search service integrated into workflow ofMicrosoft Security Response Center (MSRC)MicrosoftTechnet Blog about XIAO:We wanted to be sure to address the vulnerable code wherever it appearedacross the Microsoft code base.To that end, we have been working withMicrosoft Research to develop a “Cloned Code Detection” system that we canrun for every MSRC case to find any instance of the vulnerablecode in any shipping product.This system is the one that found several ofthe copies of CVE-2011-3402 that we are now addressing with MS12-034.
    16. 16. 19XIAO enables code clone analysis withHigh scalability, High compatibilityHigh tunability: what you tune is what you getHigh explorability:1. Clone navigation based on source tree hierarchy2. Pivoting of folder level statistics3. Folder level statistics4. Clone function list in selected folder5. Clone function filters6. Sorting by bug or refactoring potential7. Tagging1 2 3 4 5 671. Block correspondence2. Block types3. Block navigation4. Copying5. Bug filing6. Tagging1234165How to navigate through the largenumber of detected clones? How to quickly review a pair of clones?
    17. 17.  50 years of automated debugging research N papers  only 5 evaluated with actual programmers“”Chris Parnin and Alessandro Orso. Are automated debugging techniques actually helpingprogrammers?. In Proc. ISSTA 2011
    18. 18.  Academia Tend to leave human out of loop (involving human makesevaluations difficult to conduct or write) Tend not to spend effort on improving tool usability▪ tool usability would be valued more in HCI than in SE▪ too much to include both the approach/tool itself and usability/its evaluationin a single paper Real-world Often has human in the loop (familiar IDE integration, socialeffect, lack of expertise/willingness to write specs,…) Examples Agitar [ISSTA 2006] vs. Daikon [TSE 2001] Test generation in Pex based on constraint solving
    19. 19.  Goal: to identify the future directions in researchin formal methods and its transition to industrialpractice. The workshop will bring together researchers andidentify primary challenges in the field, bothfoundational, infrastructural, and in transitioningideas from research labs to developer tools.http://goto.ucsd.edu/~rjhala/NSFWorkshop/
    20. 20.  “Lack of education amongst practitioners” “Education of students in logic and design forverification” “Expertise required to create and use a verificationtool. E.g., both Astre for Airbus and SDV forWindows drivers were closely shepherded byverification experts.” “Tools require lots of up-front effort (e.g., to writespecifications)” “User effort required to guide verification tools,such as assertions or specifications”
    21. 21.  “Not integrated with standard development flows(testing)” “Too many false positives and no ranking of errors” “General usability of tools, in terms of false alarmsand error messages.The Coverity CACM paperpointed out that they had developed features thatthey do not deploy because they baffle users.Many tools choose unsoundness over soundness toavoid false alarms.”
    22. 22.  “The necessity of detailed specifications andcomplex interaction with tools, which is very costlyand discouraging for industrial, who lack high-levelspecialists.” “Feedback to users. It’s difficult to explain to userswhy automated verification tools are failing.Counterexamples to properties can be very difficultfor users to understand, especially when they areabstract, or based on incomplete environmentmodels or constraints.”
    23. 23. 2010 Dagstuhl Seminar 10111Practical Software Testing: Tool Automation and Human Factorshttp://www.dagstuhl.de/programm/kalender/semhp/?semnr=1011
    24. 24. 2010 Dagstuhl Seminar 10111Practical Software Testing: Tool Automation and Human FactorsHuman Factorshttp://www.dagstuhl.de/programm/kalender/semhp/?semnr=1011
    25. 25. Andy Ko and Brad Myers. Debugging Reinvented: Asking and Answering Why and Why NotQuestions about Program Behavior. In Proc. ICSE 2008
    26. 26.  Don’t forget human intelligence Using your tools as end-to-end solutions Helping your tools Don’t forget cooperations of human and toolintelligence; human and human intelligence Human can help your tools too Human and human could work together to help yourtools, e.g., crowdsourcing29
    27. 27.  Motivation Architecture recovery is challenging (abstraction gap) Human typically has high-level view in mind Repeat Human: define/update high-level model of interest Tool: extract a source model Human: define/update declarative mapping betweenhigh-level model and source model Tool: compute a software reflexion model Human: interpret the software reflexion modelUntil happyGail C. Murphy, David Notkin. Reengineering with Reflection Models: A Case Study. IEEE Computer 1997
    28. 28. Running Symbolic PathFinder ...…====================================================== resultsno errors detected====================================================== statisticselapsed time: 0:00:02states: new=4, visited=0,backtracked=4, end=2search: maxDepth=3,constraints=0choice generators: thread=1, data=2heap: gc=3, new=271, free=22instructions: 2875max memory: 81MBloaded code: classes=71, methods=884…31
    29. 29.  object-creation problems (OCP) - 65% external-method call problems (EMCP) – 27%Total block coverage achieved is 50%, lowest coverage 16%.32 Ex: Dynamic Symbolic Execution (DSE) /ConcolicTesting[Godefroid et al. 05][Sen et al. 05][Tillmann et al. 08] Instrument code to explore feasible paths Challenge: path explosion When desirablereceiver or argumentobjects are notgenerated
    30. 30. 33 A graph example fromQuickGraph library Includes two classesGraphDFSAlgorithm GraphAddVertexAddEdge: requiresboth vertices to bein graph00: class Graph { …03: public void AddVertex (Vertex v) {04: vertices.Add(v); // B1 }06: public Edge AddEdge (Vertex v1, Vertex v2) {07: if (!vertices.Contains(v1))08: throw new VNotFoundException("");09: // B210: if (!vertices.Contains(v2))11: throw new VNotFoundException("");12: // B314: Edge e = new Edge(v1, v2);15: edges.Add(e); } }//DFS:DepthFirstSearch18: class DFSAlgorithm { …23: public void Compute (Vertex s) { ...24: if (graph.GetEdges().Size() > 0) { // B425: isComputed = true;26: foreach (Edge e in graph.GetEdges()) {27: ... // B528: }29: } } } 33[OOPSLA 11]
    31. 31. 34 Test target: Cover truebranch (B4) of Line 24 Desired objectstate: graph shouldinclude at least oneedge Target sequence:Graph ag = new Graph();Vertex v1 = new Vertex(0);Vertex v2 = new Vertex(1);ag.AddVertex(v1);ag.AddVertex(v2);ag.AddEdge(v1, v2);DFSAlgorithm algo = newDFSAlgorithm(ag);algo.Compute(v1);3400: class Graph { …03: public void AddVertex (Vertex v) {04: vertices.Add(v); // B1 }06: public Edge AddEdge (Vertex v1, Vertex v2) {07: if (!vertices.Contains(v1))08: throw new VNotFoundException("");09: // B210: if (!vertices.Contains(v2))11: throw new VNotFoundException("");12: // B314: Edge e = new Edge(v1, v2);15: edges.Add(e); } }//DFS:DepthFirstSearch18: class DFSAlgorithm { …23: public void Compute (Vertex s) { ...24: if (graph.GetEdges().Size() > 0) { // B425: isComputed = true;26: foreach (Edge e in graph.GetEdges()) {27: ... // B528: }29: } } }[OOPSLA 11]
    32. 32.  object-creation problems (OCP) - 65% external-method call problems (EMCP) – 27%Total block coverage achieved is 50%, lowest coverage 16%.35 Ex: Dynamic Symbolic Execution (DSE) /ConcolicTesting[Godefroid et al. 05][Sen et al. 05][Tillmann et al. 08] Instrument code to explore feasible paths Challenge: path explosionTypically DSE instruments or exploresonly methods @ project under test;Third-party API external methods(network, I/O, ..):•too many paths•uninstrumentable
    33. 33. 36
    34. 34. Total block coverage achieved is 50%, lowest coverage 16%.37 Ex: Dynamic Symbolic Execution (DSE) /ConcolicTesting[Godefroid et al. 05][Sen et al. 05][Tillmann et al. 08] Instrument code to explore feasible paths Challenge: path explosionXusheng Xiao,Tao Xie, NikolaiTillmann, and Jonathan de Halleux. Precise Identification of Problems forStructural Test Generation. In Proc. ICSE 2011
    35. 35. 2010 Dagstuhl Seminar 10111Practical SoftwareTesting: Tool Automation and Human Factors
    36. 36.  Tackling object-creation problems Seeker [OOSPLA 11] , MSeqGen [ESEC/FSE 09]Covana [ICSE 2011], OCAT [ISSTA 10]Evacon [ASE 08], Symclat [ASE 06] Still not good enough (at least for now)!▪ Seeker (52%) > Pex/DSE (41%) > Randoop/random (26%) Tackling external-method call problems DBAppTesting [ESEC/FSE 11], [ASE 11] CloudAppTesting [IEEE Soft 12] Deal with only common environment APIs@NCSUASE
    37. 37. 40 Test target: Cover truebranch (B4) of Line 24 Desired objectstate: graph shouldinclude at least oneedge Target sequence:Graph ag = new Graph();Vertex v1 = new Vertex(0);Vertex v2 = new Vertex(1);ag.AddVertex(v1);ag.AddVertex(v2);ag.AddEdge(v1, v2);DFSAlgorithm algo = newDFSAlgorithm(ag);algo.Compute(v1);4000: class Graph { …03: public void AddVertex (Vertex v) {04: vertices.Add(v); // B1 }06: public Edge AddEdge (Vertex v1, Vertex v2) {07: if (!vertices.Contains(v1))08: throw new VNotFoundException("");09: // B210: if (!vertices.Contains(v2))11: throw new VNotFoundException("");12: // B314: Edge e = new Edge(v1, v2);15: edges.Add(e); } }//DFS:DepthFirstSearch18: class DFSAlgorithm { …23: public void Compute (Vertex s) { ...24: if (graph.GetEdges().Size() > 0) { // B425: isComputed = true;26: foreach (Edge e in graph.GetEdges()) {27: ... // B528: }29: } } }
    38. 38. Tackle object-creation problems with Factory Methods41
    39. 39. Tackle external-method call problems with Mock Methods orMethod InstrumentationMocking System.IO.File.ReadAllText42
    40. 40.  Human-Assisted Computing Driver: tool Helper: human Ex. Covana [Xiao et al. ICSE 2011] Human-Centric Computing Driver: human  Helper: tool Ex. Coding duels @Pex for FunInterfaces are important. Contents are important too!43
    41. 41.  Motivation Tools are often not powerful enough Human is good at some aspects that tools are not What difficulties does the tool face? How to communicate info to the user to get help? How does the user help the tool based on the info?44Iterations to form Feedback Loop
    42. 42.  Human-Assisted Computing Driver: tool Helper: human Ex. Covana [Xiao et al. ICSE 2011] Human-Centric Computing Driver: human  Helper: tool Ex. Coding duels @Pex for FunInterfaces are important. Contents are important too!45
    43. 43. 1,230,309 clicked Ask Pex!www.pexforfun.com46NikolaiTillmann, Jonathan De Halleux,Tao Xie, Sumit Gulwani and Judith Bishop.Teaching and Learning Programming and Software Engineering via InteractiveGaming. In Proc. ICSE 2013 SEE.
    44. 44. Secret Implementationclass Secret {public static int Puzzle(int x) {if (x <= 0) return 1;return x * Puzzle(x-1);}}Player Implementationclass Player {public static int Puzzle(int x) {return x;}}classTest {public static void Driver(int x) {if (Secret.Puzzle(x) != Player.Puzzle(x))throw new Exception(“Mismatch”);}}behaviorSecret Impl == Player Impl47
    45. 45.  Coding duels at http://www.pexforfun.com/ Brain exercising/learning while having fun Fun: iterative, adaptive/personalized, w/ win criterion Abstraction/generalization, debugging, problem solvingBrain exercising
    46. 46. http://pexforfun.com/gradsoftengObserved Benefits• Automatic Grading• Real-time Feedback (for Both Students andTeachers)• Fun Learning Experiences
    47. 47. “It really got me *excited*.The part that got me most isabout spreading interest in teaching CS: I do think that it’sREALLY great for teaching | learning!”“I used to love the first person shooters and thesatisfaction of blowing away a whole team ofNoobies playing Rainbow Six, but this is far morefun.”“I’m afraid I’ll have to constrain myself to spend just an houror so a day on this really exciting stuff, as I’m really stuffedwith work.”X
    48. 48. 52Internetclass Secret {public static int Puzzle(int x) {if (x <= 0) return 1;return x * Puzzle(x-1); } } Everyone cancontribute Coding duels Duel solutions
    49. 49. InternetPuzzle Games Made fromDifficult Constraints or Object-Creation ProblemsSupported by MSR SEIFAwardNing Chen and Sunghun Kim. Puzzle-based Automatic Testing: bringing humans into the loop bysolving puzzles. In Proc. ASE 2012
    50. 50. http://www.cs.washington.edu/verigames/
    51. 51. 55Pattern MatchingBug updateProblematicPattern RepositoryBug DatabaseTrace analysisBugfilingStackMine [Han et al. ICSE 12]Trace StorageTrace collectionInternetShi Han,Yingnong Dang, Song Ge, Dongmei Zhang, and Tao Xie. Performance Debugging in the Largevia Mining Millions of StackTraces. In Proc. ICSE 2012
    52. 52. “We believe that the MSRA tool is highly valuable andmuch more efficient for mass trace (100+ traces) analysis.For 1000 traces, we believe the tool saves us 4-6 weeks oftime to create new signatures, which is quite a significantproductivity boost.”- from Development Manager inWindowsHighly effective new issue discovery onWindows mini-hangContinuous impact on futureWindows versionsShi Han,Yingnong Dang, Song Ge, Dongmei Zhang, and Tao Xie. Performance Debugging in the Largevia Mining Millions of StackTraces. In Proc. ICSE 2012
    53. 53.  Don’t forget human intelligence Using your tools as end-to-end solutions Helping your tools Don’t forget cooperations of human and toolintelligence; human and human intelligence Human can help your tools too Human and human could work together to help yourtools, e.g., crowdsourcing57
    54. 54.  Human-Assisted Computing Human-Centric Computing Human-Human Cooperation
    55. 55.  Wonderful current/former students@NCSU ASE Collaborators, especially those from MicrosoftResearch Redmond/Asia, Peking University Colleagues who gave feedback and inspired meNSF grants CCF-0845272, CCF-0915400, CNS-0958235, ARO grant W911NF-08-1-0443, anNSA Science of Security, Lablet grant, a NIST grant, a 2011 Microsoft Research SEIFAward
    56. 56. Questions ?https://sites.google.com/site/asergrp/
    1. A particular slide catching your eye?

      Clipping is a handy way to collect important slides you want to go back to later.

    ×