0
Findability through Traceability
- A Realistic Application of Candidate Trace Links?
Markus Borg
Software Engineering Rese...
Lund University / Markus.Borg@cs.lth.se
• MSc. Computer science and
engineering
• Software engineer
(2007 - 2010)
• PhD St...
Lund University / Markus.Borg@cs.lth.se
Outline of the Presentation
• Industrial case
– Work task in safety-critical devel...
Lund University / Markus.Borg@cs.lth.se
Definitions
Lund University / Markus.Borg@cs.lth.se
Definitions
A trace link is a specified association
between a pair of software art...
Lund University / Markus.Borg@cs.lth.se
Definitions (2)
Findability is the degree to
which a system or environment
support...
Lund University / Markus.Borg@cs.lth.se
Definitions (3)
Information retrieval (IR) is
finding material (usually
documents)...
Lund University / Markus.Borg@cs.lth.se
Description of the Case
Lund University / Markus.Borg@cs.lth.se
Case Company
• Large mutli-national company
– Industrial automation
• Plan-driven ...
Lund University / Markus.Borg@cs.lth.se
Impact analysis
• Software changes should be analyzed before
implementation
• Task...
Lund University / Markus.Borg@cs.lth.se
• For changes that have impact on safety parts
– A question template must be fille...
Lund University / Markus.Borg@cs.lth.se
Document space
Pre-
study
SRS
Func.
spec.
System
design
Detailed
design
Source
cod...
Lund University / Markus.Borg@cs.lth.se
Manual trace link seeking
Search
similar
reports
Search
project
docs.
Ask
team
mem...
Lund University / Markus.Borg@cs.lth.se
Trace Recovery based on
Information Retrieval
Lund University / Markus.Borg@cs.lth.se
IR-based Trace Recovery
• Information need
To what should this
artifact be traced?...
Lund University / Markus.Borg@cs.lth.se
IR-based Trace Recovery (2)
IR-based trace
recovery tool
==
Search tool with
autom...
Lund University / Markus.Borg@cs.lth.se
Semi-automated trace link seeking
Search
similar
reports
Search
project
docs.
Ask
...
Lund University / Markus.Borg@cs.lth.se
Automation Analysis:
Method
Lund University / Markus.Borg@cs.lth.se
• Model of types and levels of human interaction
with automation
– 4 types
– 10 le...
Lund University / Markus.Borg@cs.lth.se
Types of Automation
Acquire
info
Analyze
info
Decide
action
Implement
action
Not m...
Lund University / Markus.Borg@cs.lth.se
Levels of Automation
• From no assistance at all to a fully autonomous
solution
(P...
Lund University / Markus.Borg@cs.lth.se
Types and levels of automation
Level
12345678910
Lund University / Markus.Borg@cs.lth.se
Types and levels of automation (2)
Level
12345678910
Lund University / Markus.Borg@cs.lth.se
Automation Analysis:
Results
Lund University / Markus.Borg@cs.lth.se
Type: Acquire + analyze information
• Software artifacts stored in different syste...
Lund University / Markus.Borg@cs.lth.se
Acquire + analyze information (2)
Pre-automation
• Engineers must access
different...
Lund University / Markus.Borg@cs.lth.se
Acquire + analyze information (3)
IA
Report
Reqs.
DB
Test DB
DMS
Lund University / Markus.Borg@cs.lth.se
Acquire + Analyze information (4)
Level
12345678910
Post auto
Pre auto
Lund University / Markus.Borg@cs.lth.se
Type: Decide on Action
• Candidate trace links found
• Decide which to report in t...
Lund University / Markus.Borg@cs.lth.se
Decide on action (2)
Level
12345678910
Post auto
Pre auto
Lund University / Markus.Borg@cs.lth.se
Type: Action implementation
• Report trace links in impact analysis report
Pre-aut...
Lund University / Markus.Borg@cs.lth.se
Action implementation (2)
Level
12345678910
Post auto
Pre auto
Lund University / Markus.Borg@cs.lth.se
Conclusion
Lund University / Markus.Borg@cs.lth.se
Deploying a trace recovery tool
• An IR-based trace recovery tool would
increase t...
Lund University / Markus.Borg@cs.lth.se
Deploying a trace recovery tool (2)
• Consequently
– less change involved than has...
Lund University / Markus.Borg@cs.lth.se
Thanks for your
attention!
Upcoming SlideShare
Loading in...5
×

Findability through Traceability - A Realistic Application of Candidate Trace Links?

120

Published on

Conference presentation from ENASE 2012 in Wroclaw, Poland.

Abstract: Since software development is of dynamic nature, the impact analysis is an inevitable work task. Traceability is known as one factor that supports this task, and several researchers have proposed traceability recovery tools to propose trace links in an existing system. However, these semi-automatic tools have not yet proven useful in industrial applications. Based on an established automation model, we analyzed the potential value of such a tool. We based our analysis on a pilot case study of an impact analysis process in a safety-critical development context, and argue that traceability recovery should be considered an investment in ndability. Moreover, several risks involved in an increased level of impact analysis automation are already plaguing the state-
of-practice workflow. Consequently, deploying a traceability recovery tool involves a lower degree
of change than has previously been acknowledged.

Published in: Science, Technology
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
120
On Slideshare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
2
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide
  • Textual content, document driven etc.
  • Assumption:Developers use consistent naming conventionsdocumentationsource codee-mails…
  • Assumption:Developers use consistent naming conventionsdocumentationsource codee-mails…
  • Assumption:Developers use consistent naming conventionsdocumentationsource codee-mails…
  • Textual content, document driven etc.
  • Textual content, document driven etc.
  • Textual content, document driven etc.
  • Textual content, document driven etc.
  • Textual content, document driven etc.
  • Assumption:Developers use consistent naming conventionsdocumentationsource codee-mails…
  • Assumption:Developers use consistent naming conventionsdocumentationsource codee-mails…
  • Textual content, document driven etc.
  • Textual content, document driven etc.
  • This research targets these two bullets...
  • This research targets these two bullets...
  • This research targets these two bullets...
  • This research targets these two bullets...
  • This research targets these two bullets...
  • This research targets these two bullets...
  • This research targets these two bullets...
  • This research targets these two bullets...
  • This research targets these two bullets...
  • This research targets these two bullets...
  • This research targets these two bullets...
  • This research targets these two bullets...
  • As opposed to what has been discussed before.
  • As opposed to what has been discussed before.
  • As opposed to what has been discussed before.
  • Transcript of "Findability through Traceability - A Realistic Application of Candidate Trace Links?"

    1. 1. Findability through Traceability - A Realistic Application of Candidate Trace Links? Markus Borg Software Engineering Research Group Dept. of Computer Science
    2. 2. Lund University / Markus.Borg@cs.lth.se • MSc. Computer science and engineering • Software engineer (2007 - 2010) • PhD Student Software Engineering (2010- ) • Research interests • Large-scale development • Traceability • Information management
    3. 3. Lund University / Markus.Borg@cs.lth.se Outline of the Presentation • Industrial case – Work task in safety-critical development • Previously proposed solution (>100 papers) – Decision support using semi-automated trace recovery • Impact on the case – Automation Analysis – Work flow: Pre auto. vs. post auto. • Conclusion
    4. 4. Lund University / Markus.Borg@cs.lth.se Definitions
    5. 5. Lund University / Markus.Borg@cs.lth.se Definitions A trace link is a specified association between a pair of software artifacts. Trace recovery is the process of establishing trace links among existing software artifacts. Cleland-Huang et al. (2012)
    6. 6. Lund University / Markus.Borg@cs.lth.se Definitions (2) Findability is the degree to which a system or environment supports navigation and retrieval. Morville (2005)
    7. 7. Lund University / Markus.Borg@cs.lth.se Definitions (3) Information retrieval (IR) is finding material (usually documents) of an unstructured nature (usually text) that satisfies an information need from within large collections. Manning et al. (2008)
    8. 8. Lund University / Markus.Borg@cs.lth.se Description of the Case
    9. 9. Lund University / Markus.Borg@cs.lth.se Case Company • Large mutli-national company – Industrial automation • Plan-driven development – Iterative gate-model – ~300 software engineers (Sweden and India) • Safety-critical – Regulations mandate dev. process – V&V is vital to certify product
    10. 10. Lund University / Markus.Borg@cs.lth.se Impact analysis • Software changes should be analyzed before implementation • Task integrated in issue tracker
    11. 11. Lund University / Markus.Borg@cs.lth.se • For changes that have impact on safety parts – A question template must be filled in by the engineer – 6 out of 13 questions require trace links Impact analysis (2) ”List modified code files” “Which documents need to be modified?” “Which requirements and functions need to be retested?”
    12. 12. Lund University / Markus.Borg@cs.lth.se Document space Pre- study SRS Func. spec. System design Detailed design Source code Design test spec. Test scripts Func. test spec. System test spec. IA Report
    13. 13. Lund University / Markus.Borg@cs.lth.se Manual trace link seeking Search similar reports Search project docs. Ask team member Q1. ? Q2. ? . . Qn. ? Q1. ! Q2. ! . . Qn. ! Impact analysis template Impact analysis template
    14. 14. Lund University / Markus.Borg@cs.lth.se Trace Recovery based on Information Retrieval
    15. 15. Lund University / Markus.Borg@cs.lth.se IR-based Trace Recovery • Information need To what should this artifact be traced? • Solution idea Suggest textually similar artifacts • Assumption Provides a useful starting point for tracing
    16. 16. Lund University / Markus.Borg@cs.lth.se IR-based Trace Recovery (2) IR-based trace recovery tool == Search tool with automatically executed queries Not very different from state-of- practice search solutions…
    17. 17. Lund University / Markus.Borg@cs.lth.se Semi-automated trace link seeking Search similar reports Search project docs. Ask team member Q1. ? Q2. ? . . Qn. ? Q1. ! Q2. ! . . Qn. ! Impact analysis template Impact analysis template
    18. 18. Lund University / Markus.Borg@cs.lth.se Automation Analysis: Method
    19. 19. Lund University / Markus.Borg@cs.lth.se • Model of types and levels of human interaction with automation – 4 types – 10 levels Analysis method (Parasuraman et al., 2000)
    20. 20. Lund University / Markus.Borg@cs.lth.se Types of Automation Acquire info Analyze info Decide action Implement action Not meaningful to distinguish for IR- systems (Parasuraman et al., 2000)
    21. 21. Lund University / Markus.Borg@cs.lth.se Levels of Automation • From no assistance at all to a fully autonomous solution (Parasuraman et al., 2000) Manual work Decision support Automated action, informs human Automated action, not informing human Level 1 Level 2-5 Level 6-8 Level 9-10
    22. 22. Lund University / Markus.Borg@cs.lth.se Types and levels of automation Level 12345678910
    23. 23. Lund University / Markus.Borg@cs.lth.se Types and levels of automation (2) Level 12345678910
    24. 24. Lund University / Markus.Borg@cs.lth.se Automation Analysis: Results
    25. 25. Lund University / Markus.Borg@cs.lth.se Type: Acquire + analyze information • Software artifacts stored in different systems, e.g., – Requirements Databases – Test Management Systems – Source Code Repositories • Challenges – Distributed information – Poor tool interoperability (i.e., information silos) – Finding relevant information
    26. 26. Lund University / Markus.Borg@cs.lth.se Acquire + analyze information (2) Pre-automation • Engineers must access different information systems • Engineer manually enters queries in search tools • Possibly support for Boolean search operations, filtering etc. Post-automation • Trace recovery tools must access different information systems • Tool automatically executes search queries • Possibly support for filtering
    27. 27. Lund University / Markus.Borg@cs.lth.se Acquire + analyze information (3) IA Report Reqs. DB Test DB DMS
    28. 28. Lund University / Markus.Borg@cs.lth.se Acquire + Analyze information (4) Level 12345678910 Post auto Pre auto
    29. 29. Lund University / Markus.Borg@cs.lth.se Type: Decide on Action • Candidate trace links found • Decide which to report in the impact analysis report Pre-automation • Engineers decide which trace links to report Post-automation • Engineers decide which trace links to report
    30. 30. Lund University / Markus.Borg@cs.lth.se Decide on action (2) Level 12345678910 Post auto Pre auto
    31. 31. Lund University / Markus.Borg@cs.lth.se Type: Action implementation • Report trace links in impact analysis report Pre-automation • Engineers manually fill in impact analysis report using free text Post-automation • Engineers establish trace links in trace recovery tool • Tool represents artifacts and trace links as a network Q1. ! Q2. ! . . Qn. !
    32. 32. Lund University / Markus.Borg@cs.lth.se Action implementation (2) Level 12345678910 Post auto Pre auto
    33. 33. Lund University / Markus.Borg@cs.lth.se Conclusion
    34. 34. Lund University / Markus.Borg@cs.lth.se Deploying a trace recovery tool • An IR-based trace recovery tool would increase the level of automation... Instead, other types of automation are addressed - but not mainly targeting decision making
    35. 35. Lund University / Markus.Borg@cs.lth.se Deploying a trace recovery tool (2) • Consequently – less change involved than has been acknowledged – risks related to decision making already present – should be combined with manual searching • An IR-based trace recovery tool should mainly be considered a way to increase findability – A complementary search tool rather than a semi-automatic trace tool
    36. 36. Lund University / Markus.Borg@cs.lth.se Thanks for your attention!
    1. A particular slide catching your eye?

      Clipping is a handy way to collect important slides you want to go back to later.

    ×