Agile Software Assessment — ICPC 2012
Upcoming SlideShare
Loading in...5
×
 

Like this? Share it with your network

Share

Agile Software Assessment — ICPC 2012

on

  • 1,046 views

Keynote presentation a

Keynote presentation a

Statistics

Views

Total Views
1,046
Views on SlideShare
1,046
Embed Views
0

Actions

Likes
0
Downloads
15
Comments
0

0 Embeds 0

No embeds

Accessibility

Categories

Upload Details

Uploaded via as Apple Keynote

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment
  • For ICPC 2012, Passau\n
  • he\n
  • Roadmap\n
  • \n
  • Real software systems change with time. Much of development is not new but maintenance, adaptation and extension of existing systems. But these systems are hard to understand as they tend to get more complex over time.\n
  • Although the architecture is one of the most important artifacts of a system to understand, it is not easily recoverable from code. This is because: (1) a system may have many architectures (eg layered and thin client), (2) there are many kinds of architecture static, run-time, build etc), (3) PLs do not offer any support to encode architectural constraints aside from coarse structuring and interfaces.\n
  • Especially with OO, where the code does not reflect run time behaviour well, more time is spent reading than writing, to understand code and to understand impact of change. IDEs do not well support this reading activity, since they focus on PL concepts, like editing, compiling and debugging, not architectural constraints, or user features.\n
  • Analysis tool are pretty limited to programming tasks. We have built many specialized tools, such as to mine features, extract architecture, or analyze evolution, but these tools are expensive to build. We need an IDE with better tools, and offering a better meta-tooling environment (tools to build tools quickly).\n
  • \n
  • Moose is a platform for modeling software artifacts to enable software analysis.\nMoose has been developed for well over a decade. It is the work of dozens of researchers, and has been the basis of numerous academic and industrial projects.\n
  • The key to Moose’s flexibility is its extensible metamodel.\nEvery tool that builds on Moose is able to extend the metamodel with concepts that it needs, such as history.\n
  • The key to Moose’s flexibility is its extensible metamodel.\nEvery tool that builds on Moose is able to extend the metamodel with concepts that it needs, such as history.\n
  • The key to Moose’s flexibility is its extensible metamodel.\nEvery tool that builds on Moose is able to extend the metamodel with concepts that it needs, such as history.\n
  • The key to Moose’s flexibility is its extensible metamodel.\nEvery tool that builds on Moose is able to extend the metamodel with concepts that it needs, such as history.\n
  • The key to Moose’s flexibility is its extensible metamodel.\nEvery tool that builds on Moose is able to extend the metamodel with concepts that it needs, such as history.\n
  • The key to Moose’s flexibility is its extensible metamodel.\nEvery tool that builds on Moose is able to extend the metamodel with concepts that it needs, such as history.\n
  • System complexity - Clone evolution view\nClass blueprint - Topic Correlation Matrix - Distribution Map for topics spread over classes in packages\nHierarchy Evolution view - Ownership Map\n
  • System complexity - Clone evolution view\nClass blueprint - Topic Correlation Matrix - Distribution Map for topics spread over classes in packages\nHierarchy Evolution view - Ownership Map\n
  • System complexity - Clone evolution view\nClass blueprint - Topic Correlation Matrix - Distribution Map for topics spread over classes in packages\nHierarchy Evolution view - Ownership Map\n
  • System complexity - Clone evolution view\nClass blueprint - Topic Correlation Matrix - Distribution Map for topics spread over classes in packages\nHierarchy Evolution view - Ownership Map\n
  • System complexity - Clone evolution view\nClass blueprint - Topic Correlation Matrix - Distribution Map for topics spread over classes in packages\nHierarchy Evolution view - Ownership Map\n
  • System complexity - Clone evolution view\nClass blueprint - Topic Correlation Matrix - Distribution Map for topics spread over classes in packages\nHierarchy Evolution view - Ownership Map\n
  • System complexity - Clone evolution view\nClass blueprint - Topic Correlation Matrix - Distribution Map for topics spread over classes in packages\nHierarchy Evolution view - Ownership Map\n
  • System complexity - Clone evolution view\nClass blueprint - Topic Correlation Matrix - Distribution Map for topics spread over classes in packages\nHierarchy Evolution view - Ownership Map\n
  • \n
  • Mondrian is a tool to script Moose visualizations using embedded domain-specific language. Essentially it is a component framework for visualization with a fluent interface.\n
  • Glamour is another such DSL, but for building dedicated model browsers.\n
  • \n
  • \n
  • The key bottleneck to assessment is creating a suitable model for analysis. If a tool does not already exist, it can take days, weeks or months to parse source files and generate models.\n
  • Developing a parser for a new language is a big challenge. Parsers may be hard to scavenge from existing tools.\nNot only source code, but other sources of information, like bug reports and emails can be invaluable for model building.\nFew projects today are built using a single language. Often a GPL is mixed with scripting languages, or even home-brewed DSLs.\n
  • Developing a parser for a new language is a big challenge. Parsers may be hard to scavenge from existing tools.\nNot only source code, but other sources of information, like bug reports and emails can be invaluable for model building.\nFew projects today are built using a single language. Often a GPL is mixed with scripting languages, or even home-brewed DSLs.\n
  • Developing a parser for a new language is a big challenge. Parsers may be hard to scavenge from existing tools.\nNot only source code, but other sources of information, like bug reports and emails can be invaluable for model building.\nFew projects today are built using a single language. Often a GPL is mixed with scripting languages, or even home-brewed DSLs.\n
  • Developing a parser for a new language is a big challenge. Parsers may be hard to scavenge from existing tools.\nNot only source code, but other sources of information, like bug reports and emails can be invaluable for model building.\nFew projects today are built using a single language. Often a GPL is mixed with scripting languages, or even home-brewed DSLs.\n
  • On the plus side, lots of sample code exists, a full parser may not be needed, and a coarse model can suffice to get started. Example mappings from code samples can be used to generate fact extractors (see previous work). Island grammars distinguish artifacts of interest (islands) from the details (sea). These can be refined when needed to capture more detail of interest. Languages can be classified into groups with similar features. This could perhaps be exploited to rapidly build fact extractors from reusable parts. Other hints link indentation and keyword recognition could be used to detect structure.\n
  • On the plus side, lots of sample code exists, a full parser may not be needed, and a coarse model can suffice to get started. Example mappings from code samples can be used to generate fact extractors (see previous work). Island grammars distinguish artifacts of interest (islands) from the details (sea). These can be refined when needed to capture more detail of interest. Languages can be classified into groups with similar features. This could perhaps be exploited to rapidly build fact extractors from reusable parts. Other hints link indentation and keyword recognition could be used to detect structure.\n
  • On the plus side, lots of sample code exists, a full parser may not be needed, and a coarse model can suffice to get started. Example mappings from code samples can be used to generate fact extractors (see previous work). Island grammars distinguish artifacts of interest (islands) from the details (sea). These can be refined when needed to capture more detail of interest. Languages can be classified into groups with similar features. This could perhaps be exploited to rapidly build fact extractors from reusable parts. Other hints link indentation and keyword recognition could be used to detect structure.\n
  • On the plus side, lots of sample code exists, a full parser may not be needed, and a coarse model can suffice to get started. Example mappings from code samples can be used to generate fact extractors (see previous work). Island grammars distinguish artifacts of interest (islands) from the details (sea). These can be refined when needed to capture more detail of interest. Languages can be classified into groups with similar features. This could perhaps be exploited to rapidly build fact extractors from reusable parts. Other hints link indentation and keyword recognition could be used to detect structure.\n
  • On the plus side, lots of sample code exists, a full parser may not be needed, and a coarse model can suffice to get started. Example mappings from code samples can be used to generate fact extractors (see previous work). Island grammars distinguish artifacts of interest (islands) from the details (sea). These can be refined when needed to capture more detail of interest. Languages can be classified into groups with similar features. This could perhaps be exploited to rapidly build fact extractors from reusable parts. Other hints link indentation and keyword recognition could be used to detect structure.\n
  • Custom analyses require custom tools. Building a tool should be as easy as writing a query in SQL or a form-based interface.\n
  • Developers are often forced to use whatever tools are at hand to address assessment tasks. Few researchers have studied developers needs.\nTools for querying, navigation, metrics computation, visualization etc need to play nicely together. This implies a need for a unifying meta-model.\nMeta-tools are tools for building tools. Should these be DSLs, GPLs, graphical tools, or what?\n
  • Developers are often forced to use whatever tools are at hand to address assessment tasks. Few researchers have studied developers needs.\nTools for querying, navigation, metrics computation, visualization etc need to play nicely together. This implies a need for a unifying meta-model.\nMeta-tools are tools for building tools. Should these be DSLs, GPLs, graphical tools, or what?\n
  • Developers are often forced to use whatever tools are at hand to address assessment tasks. Few researchers have studied developers needs.\nTools for querying, navigation, metrics computation, visualization etc need to play nicely together. This implies a need for a unifying meta-model.\nMeta-tools are tools for building tools. Should these be DSLs, GPLs, graphical tools, or what?\n
  • We need more work on understanding developers needs.\n
  • We need more work on understanding developers needs.\n
  • We need more work on understanding developers needs.\n
  • \n
  • Niko's vision. Needs more than just finding similar code snippets\n
  • How do we know the answer is out there? Whom do we ask?\n
  • How do we know the answer is out there? Whom do we ask?\n
  • How do we know the answer is out there? Whom do we ask?\n
  • Clone analysis techniques can be used to find similar code snippets relevant to the task at hand.\nEcosystem analysis tracks the evolution of related software projects.\nBig software data, on the other hand, looks at unrelated projects with similar characteristics.\nUsing this information, the IDE should be able to make useful recommendations for the task at hand by pointing to possible test cases, related bugs ...\n
  • Clone analysis techniques can be used to find similar code snippets relevant to the task at hand.\nEcosystem analysis tracks the evolution of related software projects.\nBig software data, on the other hand, looks at unrelated projects with similar characteristics.\nUsing this information, the IDE should be able to make useful recommendations for the task at hand by pointing to possible test cases, related bugs ...\n
  • Clone analysis techniques can be used to find similar code snippets relevant to the task at hand.\nEcosystem analysis tracks the evolution of related software projects.\nBig software data, on the other hand, looks at unrelated projects with similar characteristics.\nUsing this information, the IDE should be able to make useful recommendations for the task at hand by pointing to possible test cases, related bugs ...\n
  • Clone analysis techniques can be used to find similar code snippets relevant to the task at hand.\nEcosystem analysis tracks the evolution of related software projects.\nBig software data, on the other hand, looks at unrelated projects with similar characteristics.\nUsing this information, the IDE should be able to make useful recommendations for the task at hand by pointing to possible test cases, related bugs ...\n
  • Clone analysis techniques can be used to find similar code snippets relevant to the task at hand.\nEcosystem analysis tracks the evolution of related software projects.\nBig software data, on the other hand, looks at unrelated projects with similar characteristics.\nUsing this information, the IDE should be able to make useful recommendations for the task at hand by pointing to possible test cases, related bugs ...\n
  • \n
  • Mircea’s vision\nLarge software systems are so complex that one can never be sure until integration whether certain changes can have catastrophic effects at a distance.\n
  • Mircea’s vision\nLarge software systems are so complex that one can never be sure until integration whether certain changes can have catastrophic effects at a distance.\n
  • Mircea’s vision\nLarge software systems are so complex that one can never be sure until integration whether certain changes can have catastrophic effects at a distance.\n
  • Mircea’s vision\nLarge software systems are so complex that one can never be sure until integration whether certain changes can have catastrophic effects at a distance.\n
  • Continuous Assessment and Awareness\nHow is the big picture evolving?\ncf SoftwareNaut\ncf CodeMaps\n
  • Continuous Assessment and Awareness\nHow is the big picture evolving?\ncf SoftwareNaut\ncf CodeMaps\n
  • Continuous Assessment and Awareness\nHow is the big picture evolving?\ncf SoftwareNaut\ncf CodeMaps\n
  • Continuous Assessment and Awareness\nHow is the big picture evolving?\ncf SoftwareNaut\ncf CodeMaps\n
  • Continuous Assessment and Awareness\nHow is the big picture evolving?\ncf SoftwareNaut\ncf CodeMaps\n
  • Architecture is not just about layers. Need ways to specify and track architectural constraints as a project progresses.\nBy monitoring an ecosystem we can track usages of components, expose implicit dependencies, and provide better recommendations to developers.\n
  • Architecture is not just about layers. Need ways to specify and track architectural constraints as a project progresses.\nBy monitoring an ecosystem we can track usages of components, expose implicit dependencies, and provide better recommendations to developers.\n
  • The IDE does not do an adequate job of supporting developers in decision-making tasks.\nWe need to better understand needs of developers and to offer a better meta-tooling environment to support customization, context and continuous assessment.\n
  • The IDE does not do an adequate job of supporting developers in decision-making tasks.\nWe need to better understand needs of developers and to offer a better meta-tooling environment to support customization, context and continuous assessment.\n
  • The IDE does not do an adequate job of supporting developers in decision-making tasks.\nWe need to better understand needs of developers and to offer a better meta-tooling environment to support customization, context and continuous assessment.\n
  • The IDE does not do an adequate job of supporting developers in decision-making tasks.\nWe need to better understand needs of developers and to offer a better meta-tooling environment to support customization, context and continuous assessment.\n

Agile Software Assessment — ICPC 2012 Presentation Transcript

  • 1. Agile Software Assessment Oscar Nierstrasz Software Composition Group scg.unibe.ch ICPC 2012
  • 2. SCG Present and Past 2
  • 3. Agile Software AssessmentMotivation Challenges Agility in Moose
  • 4. The need for AgileSoftware Assessment
  • 5. Legacy code is hard to understand 5
  • 6. The architecture ... is not in the code 6
  • 7. Developers spend more time reading than writing code 7
  • 8. Specialized analysesrequire custom tools 8
  • 9. Agility in Moose
  • 10. Moose is a platform forsoftware and data analysis www.moosetechnology.org 10
  • 11. Model repository Nierstrasz et al. The Storyof Moose. ESEC/FSE 2005 11
  • 12. Model repository Navigation Metrics Querying Grouping Nierstrasz et al. The Storyof Moose. ESEC/FSE 2005 11
  • 13. Model repository Navigation Metrics Querying Grouping Smalltalk Nierstrasz et al. The Storyof Moose. ESEC/FSE 2005 11
  • 14. ConAn Van Hapax ... CodeCrawler Model repository Navigation Metrics Querying Grouping Smalltalk Nierstrasz et al. The Story of Moose. ESEC/FSE 2005 11
  • 15. ConAn Van Hapax ... CodeCrawler Extensible meta model Model repository Navigation Metrics Querying Grouping Smalltalk Nierstrasz et al. The Story of Moose. ESEC/FSE 2005 11
  • 16. ConAn Van Hapax ... CodeCrawlerSmalltalk Extensible meta model Java Model repositoryCOBOL Navigation C++ Metrics … Querying Grouping Smalltalk Nierstrasz et al. The Story of Moose. ESEC/FSE 2005 11
  • 17. ConAn Van Hapax ... CodeCrawlerSmalltalk Extensible meta model Java Model repositoryCOBOL External CDIF Navigation Parser C++ Metrics … Querying Grouping XMI Smalltalk Nierstrasz et al. The Story of Moose. ESEC/FSE 2005 11
  • 18. System complexity Lanza et al. Polymetric Views. TSE 2003 12
  • 19. Clone evolution Balint et al. How Developers Copy. ICPC 2006 13
  • 20. Class blueprintLanza et al. A Categorization of Classes based on the Visualization of their Internal Structure: the Class Blueprint. OOPSLA 2001 14
  • 21. Topic correlation matrix Kuhn et al. Semantic Clustering: Identifying Topics in Source Code. IST 2007 15
  • 22. Distribution map(topics spread over classes in packages) Ducasse, et al. Distribution Map. ICSM 2006 16
  • 23. Hierarchy evolution view Gîrba et al. Modeling History to Analyze Software Evolution. JSME 2006 17
  • 24. Ownership map Gîrba et al. How Developers Drive Software Evolution. IWPSE 2005 18
  • 25. 19
  • 26. Moose Demo Demo: finding deprecated classes that are still in use ...
  • 27. Mondrian Demo Demo: visualizing name cohesion within packages Meyer et al. Mondrian: An Agile Visualization Framework. SoftVis 2006
  • 28. Glamour Demo Demo: a package browser for name cohesion Bunge et al. Scripting Browsers with Glamour. ESUG 2009
  • 29. Challenges for Agile Software AssessmentCustomization Context Continuous Assessment
  • 30. Customization
  • 31. ChallengeLoad the model in the morning, analyze it in the afternoon 25
  • 32. Problems 26
  • 33. Problems Unknown languages 26
  • 34. Problems Unknown languages Unstructured text 26
  • 35. Problems Unknown languages Heterogeneous projects Perin et al. Recovery and Analysis of Transaction Scope from Scattered Information Unstructured text in Java Enterprise Applications. ICSM 2010 26
  • 36. Ideas 27
  • 37. Ideas Source code parse 5 Nierstrasz et al. Example- specify examples import Parser Driven Reconstruction of 2 1 5 Software Models. CSMR 2007 4 generate ... := ... ... ... ... infer | ... ... ... ... | ... ... ... ... 3 ... := ... ... ... ... Example mappings | ... ... ... ... | ... ... ... ... ... := ... ... ... ... | ... ... ... ... | ... ... ... ... export 6 Grammar Exploit example Model m..n 1 mappings to generate fact extractors 27
  • 38. Ideas Source code parse 5 Nierstrasz et al. Example- specify examples import Parser Driven Reconstruction of 2 1 5 Software Models. CSMR 2007 4 generate ... := ... ... ... ... infer | ... ... ... ... | ... ... ... ... 3 ... := ... ... ... ... Example mappings | ... ... ... ... | ... ... ... ... ... := ... ... ... ... | ... ... ... ... | ... ... ... ... export 6 Grammar Exploit example Model m..n 1 mappings to generate fact extractors Incrementally refine island grammars 27
  • 39. Ideas Source code parse 5 Nierstrasz et al. Example- specify examples import Parser Driven Reconstruction of 2 1 5 Software Models. CSMR 2007 4 generate ... := ... ... ... ... infer | ... ... ... ... | ... ... ... ... 3 ... := ... ... ... ... Example mappings | ... ... ... ... | ... ... ... ... ... := ... ... ... ... | ... ... ... ... | ... ... ... ... export 6 Grammar Exploit example Model m..n 1 mappings to generate fact extractors Incrementally refine island grammars Exploit eg indentation as a proxy for structure 27
  • 40. Ideas Source code parse 5 Nierstrasz et al. Example- specify examples import Parser Driven Reconstruction of 2 1 5 Software Models. CSMR 2007 4 generate ... := ... ... ... ... infer | ... ... ... ... | ... ... ... ... 3 ... := ... ... ... ... Example mappings | ... ... ... ... | ... ... ... ... ... := ... ... ... ... | ... ... ... ... | ... ... ... ... export 6 Grammar Exploit example Model m..n 1 mappings to generate fact extractors Exploit similarities between languages Incrementally (adapt and compose) refine island grammars Exploit eg indentation as a proxy for structure 27
  • 41. Challenge Build a new assessment tool in ten minutes 28
  • 42. Problems 29
  • 43. Problems What tools do developers really need? 29
  • 44. Problems What tools do developers really need? What is a unifying meta-model for tool construction? 29
  • 45. Problems What tools do developers really need? What is a unifying meta-model for tool What are appropriate construction? meta-tools? 29
  • 46. Ideas 30
  • 47. Ideas Analyze developer needs (!) 30
  • 48. Ideas Analyze developer needs (!)From meta-models to interactive DSLs 30
  • 49. Ideas Analyze developer needs (!) “Malleable”IDEFrom meta-models to (not just plug-ins) interactive DSLs 30
  • 50. Context
  • 51. Challenge “Who had this problem before, and how did they solve it? 32
  • 52. Problems 33
  • 53. Problems Is the answer out there? 33
  • 54. Problems Is the answer out there? How to express the query? 33
  • 55. Problems Is the answer out there? How to express the query? Is intent captured? 33
  • 56. Ideas 34
  • 57. IdeasClone analysis for querying 34
  • 58. Ideas Ecosystem analysisClone analysis for querying 34
  • 59. Ideas Ecosystem analysisClone analysis for querying Exploit big software data Lungu et al. Big Software Data Analysis. ERCIM News 2012 34
  • 60. Ideas Ecosystem analysisClone analysis for querying Exploit big software data Embedding Lungu et al. Big Software Data intelligence in Analysis. ERCIM News 2012 the IDE 34
  • 61. Continuous Assessment
  • 62. Challenge “What will my code change impact?” 36
  • 63. Challenge “What will my code change impact?” 36
  • 64. Problems 37
  • 65. Problems Multi-Version Analysis Tests Analysis Softwarenaut-Core Understanding architectural constraints Lungu et al. Evolutionary and Collaborative Software ArchitectureRecovery with Softwarenaut. SCP 2012 37
  • 66. Problems Multi-Version Analysis Tests Implicit dependencies Analysis Softwarenaut-Core Understanding architectural constraints Lungu et al. Evolutionary and Collaborative Software ArchitectureRecovery with Softwarenaut. SCP 2012 37
  • 67. Problems Multi-Version Analysis Tests Implicit dependencies Analysis Softwarenaut-Core Understanding architectural constraints Ripple effects Lungu et al. Evolutionary and Robbes et al. A Study of Ripple Effects in Collaborative Software Architecture Software Ecosystems. ICSE-NIER 2011Recovery with Softwarenaut. SCP 2012 37
  • 68. Ideas 38
  • 69. Ideas Architecture monitoring (beyond layers) 38
  • 70. Ideas Architecture monitoring (beyond layers) Ecosystem monitoring 38
  • 71. Conclusion Current IDEs offer developers poor support for software assessment
  • 72. Conclusion Current IDEs offer developers poor support for software assessment Developers need support for customization, context, continuous assessment