Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Findability through Traceability
- A Realistic Application of Candidate Trace Links?
Markus Borg
Software Engineering Rese...
Lund University / Markus.Borg@cs.lth.se
• MSc. Computer science and
engineering
• Software engineer
(2007 - 2010)
• PhD St...
Lund University / Markus.Borg@cs.lth.se
Outline of the Presentation
• Industrial case
– Work task in safety-critical devel...
Lund University / Markus.Borg@cs.lth.se
Definitions
Lund University / Markus.Borg@cs.lth.se
Definitions
A trace link is a specified association
between a pair of software art...
Lund University / Markus.Borg@cs.lth.se
Definitions (2)
Findability is the degree to
which a system or environment
support...
Lund University / Markus.Borg@cs.lth.se
Definitions (3)
Information retrieval (IR) is
finding material (usually
documents)...
Lund University / Markus.Borg@cs.lth.se
Description of the Case
Lund University / Markus.Borg@cs.lth.se
Case Company
• Large mutli-national company
– Industrial automation
• Plan-driven ...
Lund University / Markus.Borg@cs.lth.se
Impact analysis
• Software changes should be analyzed before
implementation
• Task...
Lund University / Markus.Borg@cs.lth.se
• For changes that have impact on safety parts
– A question template must be fille...
Lund University / Markus.Borg@cs.lth.se
Document space
Pre-
study
SRS
Func.
spec.
System
design
Detailed
design
Source
cod...
Lund University / Markus.Borg@cs.lth.se
Manual trace link seeking
Search
similar
reports
Search
project
docs.
Ask
team
mem...
Lund University / Markus.Borg@cs.lth.se
Trace Recovery based on
Information Retrieval
Lund University / Markus.Borg@cs.lth.se
IR-based Trace Recovery
• Information need
To what should this
artifact be traced?...
Lund University / Markus.Borg@cs.lth.se
IR-based Trace Recovery (2)
IR-based trace
recovery tool
==
Search tool with
autom...
Lund University / Markus.Borg@cs.lth.se
Semi-automated trace link seeking
Search
similar
reports
Search
project
docs.
Ask
...
Lund University / Markus.Borg@cs.lth.se
Automation Analysis:
Method
Lund University / Markus.Borg@cs.lth.se
• Model of types and levels of human interaction
with automation
– 4 types
– 10 le...
Lund University / Markus.Borg@cs.lth.se
Types of Automation
Acquire
info
Analyze
info
Decide
action
Implement
action
Not m...
Lund University / Markus.Borg@cs.lth.se
Levels of Automation
• From no assistance at all to a fully autonomous
solution
(P...
Lund University / Markus.Borg@cs.lth.se
Types and levels of automation
Level
12345678910
Lund University / Markus.Borg@cs.lth.se
Types and levels of automation (2)
Level
12345678910
Lund University / Markus.Borg@cs.lth.se
Automation Analysis:
Results
Lund University / Markus.Borg@cs.lth.se
Type: Acquire + analyze information
• Software artifacts stored in different syste...
Lund University / Markus.Borg@cs.lth.se
Acquire + analyze information (2)
Pre-automation
• Engineers must access
different...
Lund University / Markus.Borg@cs.lth.se
Acquire + analyze information (3)
IA
Report
Reqs.
DB
Test DB
DMS
Lund University / Markus.Borg@cs.lth.se
Acquire + Analyze information (4)
Level
12345678910
Post auto
Pre auto
Lund University / Markus.Borg@cs.lth.se
Type: Decide on Action
• Candidate trace links found
• Decide which to report in t...
Lund University / Markus.Borg@cs.lth.se
Decide on action (2)
Level
12345678910
Post auto
Pre auto
Lund University / Markus.Borg@cs.lth.se
Type: Action implementation
• Report trace links in impact analysis report
Pre-aut...
Lund University / Markus.Borg@cs.lth.se
Action implementation (2)
Level
12345678910
Post auto
Pre auto
Lund University / Markus.Borg@cs.lth.se
Conclusion
Lund University / Markus.Borg@cs.lth.se
Deploying a trace recovery tool
• An IR-based trace recovery tool would
increase t...
Lund University / Markus.Borg@cs.lth.se
Deploying a trace recovery tool (2)
• Consequently
– less change involved than has...
Lund University / Markus.Borg@cs.lth.se
Thanks for your
attention!
Upcoming SlideShare
Loading in …5
×

Findability through Traceability - A Realistic Application of Candidate Trace Links?

660 views

Published on

Conference presentation from ENASE 2012 in Wroclaw, Poland.

Abstract: Since software development is of dynamic nature, the impact analysis is an inevitable work task. Traceability is known as one factor that supports this task, and several researchers have proposed traceability recovery tools to propose trace links in an existing system. However, these semi-automatic tools have not yet proven useful in industrial applications. Based on an established automation model, we analyzed the potential value of such a tool. We based our analysis on a pilot case study of an impact analysis process in a safety-critical development context, and argue that traceability recovery should be considered an investment in ndability. Moreover, several risks involved in an increased level of impact analysis automation are already plaguing the state-
of-practice workflow. Consequently, deploying a traceability recovery tool involves a lower degree
of change than has previously been acknowledged.

Published in: Science, Technology
  • Be the first to comment

Findability through Traceability - A Realistic Application of Candidate Trace Links?

  1. 1. Findability through Traceability - A Realistic Application of Candidate Trace Links? Markus Borg Software Engineering Research Group Dept. of Computer Science
  2. 2. Lund University / Markus.Borg@cs.lth.se • MSc. Computer science and engineering • Software engineer (2007 - 2010) • PhD Student Software Engineering (2010- ) • Research interests • Large-scale development • Traceability • Information management
  3. 3. Lund University / Markus.Borg@cs.lth.se Outline of the Presentation • Industrial case – Work task in safety-critical development • Previously proposed solution (>100 papers) – Decision support using semi-automated trace recovery • Impact on the case – Automation Analysis – Work flow: Pre auto. vs. post auto. • Conclusion
  4. 4. Lund University / Markus.Borg@cs.lth.se Definitions
  5. 5. Lund University / Markus.Borg@cs.lth.se Definitions A trace link is a specified association between a pair of software artifacts. Trace recovery is the process of establishing trace links among existing software artifacts. Cleland-Huang et al. (2012)
  6. 6. Lund University / Markus.Borg@cs.lth.se Definitions (2) Findability is the degree to which a system or environment supports navigation and retrieval. Morville (2005)
  7. 7. Lund University / Markus.Borg@cs.lth.se Definitions (3) Information retrieval (IR) is finding material (usually documents) of an unstructured nature (usually text) that satisfies an information need from within large collections. Manning et al. (2008)
  8. 8. Lund University / Markus.Borg@cs.lth.se Description of the Case
  9. 9. Lund University / Markus.Borg@cs.lth.se Case Company • Large mutli-national company – Industrial automation • Plan-driven development – Iterative gate-model – ~300 software engineers (Sweden and India) • Safety-critical – Regulations mandate dev. process – V&V is vital to certify product
  10. 10. Lund University / Markus.Borg@cs.lth.se Impact analysis • Software changes should be analyzed before implementation • Task integrated in issue tracker
  11. 11. Lund University / Markus.Borg@cs.lth.se • For changes that have impact on safety parts – A question template must be filled in by the engineer – 6 out of 13 questions require trace links Impact analysis (2) ”List modified code files” “Which documents need to be modified?” “Which requirements and functions need to be retested?”
  12. 12. Lund University / Markus.Borg@cs.lth.se Document space Pre- study SRS Func. spec. System design Detailed design Source code Design test spec. Test scripts Func. test spec. System test spec. IA Report
  13. 13. Lund University / Markus.Borg@cs.lth.se Manual trace link seeking Search similar reports Search project docs. Ask team member Q1. ? Q2. ? . . Qn. ? Q1. ! Q2. ! . . Qn. ! Impact analysis template Impact analysis template
  14. 14. Lund University / Markus.Borg@cs.lth.se Trace Recovery based on Information Retrieval
  15. 15. Lund University / Markus.Borg@cs.lth.se IR-based Trace Recovery • Information need To what should this artifact be traced? • Solution idea Suggest textually similar artifacts • Assumption Provides a useful starting point for tracing
  16. 16. Lund University / Markus.Borg@cs.lth.se IR-based Trace Recovery (2) IR-based trace recovery tool == Search tool with automatically executed queries Not very different from state-of- practice search solutions…
  17. 17. Lund University / Markus.Borg@cs.lth.se Semi-automated trace link seeking Search similar reports Search project docs. Ask team member Q1. ? Q2. ? . . Qn. ? Q1. ! Q2. ! . . Qn. ! Impact analysis template Impact analysis template
  18. 18. Lund University / Markus.Borg@cs.lth.se Automation Analysis: Method
  19. 19. Lund University / Markus.Borg@cs.lth.se • Model of types and levels of human interaction with automation – 4 types – 10 levels Analysis method (Parasuraman et al., 2000)
  20. 20. Lund University / Markus.Borg@cs.lth.se Types of Automation Acquire info Analyze info Decide action Implement action Not meaningful to distinguish for IR- systems (Parasuraman et al., 2000)
  21. 21. Lund University / Markus.Borg@cs.lth.se Levels of Automation • From no assistance at all to a fully autonomous solution (Parasuraman et al., 2000) Manual work Decision support Automated action, informs human Automated action, not informing human Level 1 Level 2-5 Level 6-8 Level 9-10
  22. 22. Lund University / Markus.Borg@cs.lth.se Types and levels of automation Level 12345678910
  23. 23. Lund University / Markus.Borg@cs.lth.se Types and levels of automation (2) Level 12345678910
  24. 24. Lund University / Markus.Borg@cs.lth.se Automation Analysis: Results
  25. 25. Lund University / Markus.Borg@cs.lth.se Type: Acquire + analyze information • Software artifacts stored in different systems, e.g., – Requirements Databases – Test Management Systems – Source Code Repositories • Challenges – Distributed information – Poor tool interoperability (i.e., information silos) – Finding relevant information
  26. 26. Lund University / Markus.Borg@cs.lth.se Acquire + analyze information (2) Pre-automation • Engineers must access different information systems • Engineer manually enters queries in search tools • Possibly support for Boolean search operations, filtering etc. Post-automation • Trace recovery tools must access different information systems • Tool automatically executes search queries • Possibly support for filtering
  27. 27. Lund University / Markus.Borg@cs.lth.se Acquire + analyze information (3) IA Report Reqs. DB Test DB DMS
  28. 28. Lund University / Markus.Borg@cs.lth.se Acquire + Analyze information (4) Level 12345678910 Post auto Pre auto
  29. 29. Lund University / Markus.Borg@cs.lth.se Type: Decide on Action • Candidate trace links found • Decide which to report in the impact analysis report Pre-automation • Engineers decide which trace links to report Post-automation • Engineers decide which trace links to report
  30. 30. Lund University / Markus.Borg@cs.lth.se Decide on action (2) Level 12345678910 Post auto Pre auto
  31. 31. Lund University / Markus.Borg@cs.lth.se Type: Action implementation • Report trace links in impact analysis report Pre-automation • Engineers manually fill in impact analysis report using free text Post-automation • Engineers establish trace links in trace recovery tool • Tool represents artifacts and trace links as a network Q1. ! Q2. ! . . Qn. !
  32. 32. Lund University / Markus.Borg@cs.lth.se Action implementation (2) Level 12345678910 Post auto Pre auto
  33. 33. Lund University / Markus.Borg@cs.lth.se Conclusion
  34. 34. Lund University / Markus.Borg@cs.lth.se Deploying a trace recovery tool • An IR-based trace recovery tool would increase the level of automation... Instead, other types of automation are addressed - but not mainly targeting decision making
  35. 35. Lund University / Markus.Borg@cs.lth.se Deploying a trace recovery tool (2) • Consequently – less change involved than has been acknowledged – risks related to decision making already present – should be combined with manual searching • An IR-based trace recovery tool should mainly be considered a way to increase findability – A complementary search tool rather than a semi-automatic trace tool
  36. 36. Lund University / Markus.Borg@cs.lth.se Thanks for your attention!

×