An evaluation of the pragmatics of
    web-based bug tracking tools
Proc. 2nd Intl. Conf. on the
Pragmatic Web
Tilburg, Ne...
Context of inquiry
● Why web-based tools to support software
  development?
  – Distributed environments, or at least wher...
Difficulty with tools
● Effectiveness is determined by how well
  participants understand how to use them.
● Mere use of t...
Purpose in developing models
● To better characterize and describe the
  processes themselves
● To formally analyze and ev...
Techniques
● Workflow modeling
● Software problem tracking
  – sometimes called “bug tracking”
● Conceptual graphs:
  – a ...
Workflow modeling
● Aim is to provide framework for describing and
  evaluating software development processes
● Neutral w...
Workflow step
● Starting with RENISYS
  – An organizational role has particular obligations
    to an activity, and an int...
Beyond RENISYS
● Include concept “Intention” with respect to a role in
  the process.
   – Previous models show only the o...
Software problem tracking process
● Part of software configuration management (SCM)
● Many people involved
● Issues to con...
Modeling problem resolution processes

● Bug-tracking is one kind of problem
  resolution process.
● Standard ISO/IEC 1220...
Existing tools and processes
● No explicit set of roles which are defined in the
  process
● Most tools seem implicitly ge...
Ontology for bug tracking




                    Intelligent Systems Laboratory
Model for reporting a bug




                     Intelligent Systems Laboratory
Model for fixing a bug




                     Intelligent Systems Laboratory
Validation of model graphs
● Data mining
  – Use conceptual graph tools to scan the wealth of
    existing data (e.g., Rip...
ISO/IEC 12207 process
“… all detected problems are promptly reported and
entered into the Problem Resolution Process; acti...
ISO/IEC 12207 process model




                   Intelligent Systems Laboratory
ISO/IEC 12207 process model
● What’s missing?




                    Intelligent Systems Laboratory
Bugzilla bug tracking




                        Intelligent Systems Laboratory
Bugzilla’s model
● Nothing missing!




                     Intelligent Systems Laboratory
Sourceforge bug tracking
● Assignee - The project administrator to
  which a tracker item is assigned. Can be
  chosen fro...
Status?
 ●The online help says:
   You can set the status to 'Pending' if you are
   waiting for a response from the track...
What does “support” a process mean?
● Consider the attributes of “category” and
  “group”
  – Each project administrator c...
Modeling “status”
● One educator reports:
  – “if the phrases describing subtask status are not defined,
    different stu...
Status as a derived concept
● Process perspective: item’s status reflects
  the process that produced it




             ...
Summary
● Many current tools are incomplete with respect to
  organizational needs
  – Because tools do not address the is...
Next steps
● Empirical validation of the approach.
  – compare the performance and experience of software
    engineering ...
Further steps
● Study the subsequent process of actually
  correcting the bugs identified during the process,
  with dutie...
Upcoming SlideShare
Loading in …5
×

ICPW2007.Delugach

1,287
-1

Published on

Published in: Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
1,287
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
35
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

ICPW2007.Delugach

  1. 1. An evaluation of the pragmatics of web-based bug tracking tools Proc. 2nd Intl. Conf. on the Pragmatic Web Tilburg, Netherlands 23 Oct 2007 Harry S. Delugach Intelligent Systems Laboratory Computer Science Department Univ. of Alabama in Huntsville, USA Intelligent Systems Laboratory
  2. 2. Context of inquiry ● Why web-based tools to support software development? – Distributed environments, or at least where the participants might have difficulty meeting regularly face-to-face – Web-based tools allow them to collaborate in a generally cost-effective way – Large number of activities and artifacts to be created and maintained Intelligent Systems Laboratory
  3. 3. Difficulty with tools ● Effectiveness is determined by how well participants understand how to use them. ● Mere use of tools is not sufficient to support an effective process. ● Much current work is focused on characteristics of artifacts or products ● This work focuses on the roles and purposes of the participants Intelligent Systems Laboratory
  4. 4. Purpose in developing models ● To better characterize and describe the processes themselves ● To formally analyze and evaluate tools with respect to generally accepted process models ● To formally compare and contrast the models with each other ● To arrive at a shared process model among developers Intelligent Systems Laboratory
  5. 5. Techniques ● Workflow modeling ● Software problem tracking – sometimes called “bug tracking” ● Conceptual graphs: – a convenient formalism – easily understood visualization aid to represent the models Intelligent Systems Laboratory
  6. 6. Workflow modeling ● Aim is to provide framework for describing and evaluating software development processes ● Neutral with respect to being either normative or descriptive. ● Being “informal” does not render a model incapable of being modeled ● We assume that developing a model of a process is generally more useful than not modeling it at all. Intelligent Systems Laboratory
  7. 7. Workflow step ● Starting with RENISYS – An organizational role has particular obligations to an activity, and an intention to accomplish Intelligent Systems Laboratory
  8. 8. Beyond RENISYS ● Include concept “Intention” with respect to a role in the process. – Previous models show only the obligations (required, allowed, prohibited) of a particular role. – No indication as to why a particular role would be given a particular assignment. – For example, why would a program manager be required to review a change, or why would a developer be allowed (but not required) to make a change? ● Include status with respect to that intention Intelligent Systems Laboratory
  9. 9. Software problem tracking process ● Part of software configuration management (SCM) ● Many people involved ● Issues to consider: – Who is involved? – What are stakeholders’ roles in the process’s success? – What responsibilities do they hold with respect to the system? – What are their communication needs with other stakeholders? Intelligent Systems Laboratory
  10. 10. Modeling problem resolution processes ● Bug-tracking is one kind of problem resolution process. ● Standard ISO/IEC 12207.0 clause 6.8 ● Processes supported/implied by tools – Bugzilla – Trac – sourceforge Intelligent Systems Laboratory
  11. 11. Existing tools and processes ● No explicit set of roles which are defined in the process ● Most tools seem implicitly geared toward a particular change control process ● Some of them appear to imply certain role ● Others appear role-neutral. ● Generic models for software engineering processes Intelligent Systems Laboratory
  12. 12. Ontology for bug tracking Intelligent Systems Laboratory
  13. 13. Model for reporting a bug Intelligent Systems Laboratory
  14. 14. Model for fixing a bug Intelligent Systems Laboratory
  15. 15. Validation of model graphs ● Data mining – Use conceptual graph tools to scan the wealth of existing data (e.g., Ripoche and Sansonnet) – Emails, forum posts, program source code comments identifier names in programs ● Human subjects – Paraphrasing graphs in natural language – Analyze comparison graphs Intelligent Systems Laboratory
  16. 16. ISO/IEC 12207 process “… all detected problems are promptly reported and entered into the Problem Resolution Process; action is initiated on them; relevant parties are advised of the existence of the problem as appropriate; causes are identified, analyzed, and, where possible, eliminated; resolution and disposition are achieved; status is tracked and reported…” ● Described by text in passive voice – E.g., reported by whom, to whom, who initiates action, etc. Intelligent Systems Laboratory
  17. 17. ISO/IEC 12207 process model Intelligent Systems Laboratory
  18. 18. ISO/IEC 12207 process model ● What’s missing? Intelligent Systems Laboratory
  19. 19. Bugzilla bug tracking Intelligent Systems Laboratory
  20. 20. Bugzilla’s model ● Nothing missing! Intelligent Systems Laboratory
  21. 21. Sourceforge bug tracking ● Assignee - The project administrator to which a tracker item is assigned. Can be chosen from one of the administrators registered in this project. ● Status - This is the (potentially changing) current status of a bug. ● Priority – a nine-level scale. ● Category – project-specific. ● Group – project-specific. Intelligent Systems Laboratory
  22. 22. Status? ●The online help says: You can set the status to 'Pending' if you are waiting for a response from the tracker item author. When the author responds the status is automatically reset to that of 'Open'. Otherwise, if the author doesn't respond within an admin- defined amount of time (default is 14 days) then the item is given a status of 'Deleted'. ●Suggests definitions and process Intelligent Systems Laboratory
  23. 23. What does “support” a process mean? ● Consider the attributes of “category” and “group” – Each project administrator can choose them based on their own preferences. – Automated bug tracker has no capability to: • Relate categories or groups to each other, or • To accommodate constraints between particular categories, groups or values of the other attributes (except for the ability to search the bug list by value).  For example, are “category” and “group” orthogonal to each other, or is a group a sub-category, etc.? Intelligent Systems Laboratory
  24. 24. Modeling “status” ● One educator reports: – “if the phrases describing subtask status are not defined, different student teams often give different meanings to the same phrase. Even worse, sometimes, different members in the same team would interpret the same phrase differently.” ● Need to define status phrases indicating which role and process are involved – e.g., the status “Ready for Review” meant ready to be reviewed by the quality assurance (QA) role on the team. A better way to name this would be an explicit “Ready for Review by QA” status. Intelligent Systems Laboratory
  25. 25. Status as a derived concept ● Process perspective: item’s status reflects the process that produced it Intelligent Systems Laboratory
  26. 26. Summary ● Many current tools are incomplete with respect to organizational needs – Because tools do not address the issues of why someone is authorized (or not) to make a change to an item’s status, it is likely for a bug’s status to be inconsistent with a process ● The advantages of using conceptual graphs to represent the processes – Have the potential to be formally manipulated and compared – Are well-defined as in ISO/IEC 24707 (Common Logic) – Provide an easily understood visual description of the process for developers and analysts. Intelligent Systems Laboratory
  27. 27. Next steps ● Empirical validation of the approach. – compare the performance and experience of software engineering project teams • some teams using conventional approaches and others using the pragmatically supported approach. – incorporate the ideas into an existing production-level software organization (assuming they already perform sufficient metrics on their process) – perform some linguistic analyses of various natural language used or produced in software development, especially in distributed communities where a large portion of the interaction takes place through electronic means. Intelligent Systems Laboratory
  28. 28. Further steps ● Study the subsequent process of actually correcting the bugs identified during the process, with duties and responsibilities assigned to appropriate roles – Involves a superset of the same roles involved in problem tracking. ● Compare different organizations’ processes to find common elements, and (likely) missing elements; this is a natural next step. ● Identify where processes actually conflict with each other. Intelligent Systems Laboratory

×