Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

TAO: Turing test As Objective function

1,138 views

Published on

A vision statement, presented at RAISE 2013 (Realising Synergy between AI and Software Engineering)

Published in: Technology
  • Be the first to comment

  • Be the first to like this

TAO: Turing test As Objective function

  1. 1. 26 May 2013, RAISE WorkshopTAOismShin Yoo/Centre for Research on Evolution Search and Testing, UCL, UK
  2. 2. MyWorld
  3. 3. MyWorldSoftwareEngineering
  4. 4. MyWorldMeta-heuristicOptimisationSoftwareEngineering
  5. 5. MyWorldSBSEMeta-heuristicOptimisationSoftwareEngineering
  6. 6. MyWorldSBSEMeta-heuristicOptimisationSoftwareEngineeringSBSE = Representation + Objective Function + Optimisation
  7. 7. Test Data Generation Software Modularisation
  8. 8. Test Data Generation“I want the value of x to makethe following predicate true”if(x == 3) {...}Software Modularisation
  9. 9. Test Data Generation“I want the value of x to makethe following predicate true”if(x == 3) {...}Objective Functionf(x) = |3 - x|Software Modularisation
  10. 10. Test Data Generation“I want the value of x to makethe following predicate true”if(x == 3) {...}Objective Functionf(x) = |3 - x|This is concrete: the problemdefines the objective functiondirectly and concretely.Software Modularisation
  11. 11. Test Data Generation“I want the value of x to makethe following predicate true”if(x == 3) {...}Objective Functionf(x) = |3 - x|This is concrete: the problemdefines the objective functiondirectly and concretely.Software Modularisation“I want my 892 classesto be properly modularised”(insert your ugliest classes here)
  12. 12. Test Data Generation“I want the value of x to makethe following predicate true”if(x == 3) {...}Objective Functionf(x) = |3 - x|This is concrete: the problemdefines the objective functiondirectly and concretely.Software Modularisation“I want my 892 classesto be properly modularised”(insert your ugliest classes here)Objective Function?
  13. 13. Test Data Generation“I want the value of x to makethe following predicate true”if(x == 3) {...}Objective Functionf(x) = |3 - x|This is concrete: the problemdefines the objective functiondirectly and concretely.Software Modularisation“I want my 892 classesto be properly modularised”(insert your ugliest classes here)Objective FunctionMin. coupling, Max. cohesion
  14. 14. Test Data Generation“I want the value of x to makethe following predicate true”if(x == 3) {...}Objective Functionf(x) = |3 - x|This is concrete: the problemdefines the objective functiondirectly and concretely.Software Modularisation“I want my 892 classesto be properly modularised”(insert your ugliest classes here)Objective FunctionThis is surrogate: the objectivefunction is something we hope tocorrelate with some abstractproperty.Min. coupling, Max. cohesion
  15. 15. How do we solve problems withsurrogate objectives better?
  16. 16. TAOism
  17. 17. TAOism
  18. 18. TAOism
  19. 19. TAOism
  20. 20. TAOism
  21. 21. TAOismTuring-test As Objective function
  22. 22. TAO Hypothesis✤ Surrogate objective functions are better learnt then defined.
  23. 23. A la Brooks...✤ Having a predefined representation of the problem (objectivefunction) may not produce an intelligent solution✤ It is better to use the real world (i.e. the real SE problem) as its ownrepresentational model✤ Learner should be given a complete freedom over from which tolearn the objective function
  24. 24. New ProblemInstancesTuring TestHistoricalBest PracticeAllObservablesLearntObjective
  25. 25. TAO Challenge
  26. 26. TAO Challenge✤ Software engineers moan about realistic empirical evaluation
  27. 27. TAO Challenge✤ Software engineers moan about realistic empirical evaluation✤ We interact with open source community very statically
  28. 28. TAO Challenge✤ Software engineers moan about realistic empirical evaluation✤ We interact with open source community very statically✤ Why don’t we just commit our AI/SBSE/$#@% generated feature/bug patch/test case/*#$%@ directly to open source projects?
  29. 29. TAO Challenge✤ Software engineers moan about realistic empirical evaluation✤ We interact with open source community very statically✤ Why don’t we just commit our AI/SBSE/$#@% generated feature/bug patch/test case/*#$%@ directly to open source projects?✤ Will they notice? Will they condemn us, or welcome us?
  30. 30. Summary✤ Some SE objective functions are merely surrogates✤ These are better learnt then defined; humans are the final judges✤ If we are to use AI techniques to solve SE problems, our goal shouldbe Turing Test

×