Ahab's Leg Dilemma
Upcoming SlideShare
Loading in...5
×
 

Ahab's Leg Dilemma

on

  • 725 views

Luca Sabatucci

Luca Sabatucci
Ahab's Leg Dilemma - on the Design of a Controlled Experiment

Empire 2011, Trento, Italy

Statistics

Views

Total Views
725
Views on SlideShare
725
Embed Views
0

Actions

Likes
1
Downloads
0
Comments
0

0 Embeds 0

No embeds

Accessibility

Upload Details

Uploaded via as Microsoft PowerPoint

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment
  • 1956, John Houston directed a film adaptation of the book starring Gregory Peck as Captain Ahab

Ahab's Leg Dilemma Ahab's Leg Dilemma Presentation Transcript

  • Ahabs Leg Dilemma: on the Design of aControlled Experiment Luca Sabatucci Mariano Ceccato Alessandro Marchetto Angelo Susi
  • The Ahab’s Leg dilemma• When changing media (or communication style) we need to add details to a story, to keep the story engaging Umberto Eco 1956 Ray Bradbury & John Huston.
  • The Ahab’s Leg dilemma• The peg leg is fundamental for the story – Deciding which leg is a peg one has no bearing on it – When the peg-leg is instantiated, this decision may generate of lot of consequences 1930 Warner Bros. 1956 Ray Bradbury & John Huston.Directed by Lloyd Bacon.
  • Ahab’s Leg in RE• Often, narrative scenarios are used to validate requirements with stakeholders in focus groups• Narrative scenarios are derived from requirements (change of communication style)• Details must be added during translation to instantiate generic requirements into a concrete spatial-temporal context• Stakeholders might be distracted by irrelevant details
  • An example from our experience Requirement: the system communicates with caregivers with low and high priority signals The camera PDA displays identifies Fall on that an the event staircase unknown person and sends a is fallen in the signal to staircase caregiver’s PDA Maria falls on the staircase• The focus group was proceeding well until a nurse commented on the PDA (Ahab’s Leg) – PDA is intrusive (to carry around, battery…) and it would change working practices – Lively discussion on less intrusive devices – This was pointless, because the kind of device was not yet decided Credits: picture (c) By Chiara Leonardi
  • Mandatory Vs Optional• Some are mandatory to make the story concrete and believable (e.g., PDA) – Concreteness is important for stakeholders to envisage functionalities• Some are optional and choreographic, needed just to increase the stakeholders engagement (e.g., name of the patient)• Does mandatory and optional ALs affect in the same way requirement validation sessions? The camera PDA displays identifies Fall on that an the event staircase unknown person and sends a is fallen in the signal to staircase caregiver’s PDA Maria falls on the staircase Credits: picture (c) By Chiara Leonardi
  • Cardinality• Optional ALs can be removed – Many: scenarios very concrete but with the risk of high distraction – Few: scenarios very abstract, difficult to present to stakeholders• Influence of personal and contextual background: – Melville did not specify if all the member of the crew had two legs. But the reader assumes it based on his/her common knowledge of the real world – Abstract scenarios make stakeholders mentally complete missing details • The initial scenario is corrupted • No possibility to control ALs by the analyst The camera PDA displays identifies Fall on that an the event staircase unknown person and sends a is fallen in the signal to staircase caregiver’s PDA Maria falls on the staircase
  • Stakeholder awareness• Not realistic to compare scenario with and without ALs (the latter does not make sense)• If the discussion is moderated by a facilitator, he/she could highlight irrelevant details to avoid spending time in discussing about them – Risk of attracting even more attention on them – Just mention that there are more and less important details (with some example)• Exploiting stakeholder awareness is probably the more realistic approach and analyst would take, to limit distraction.
  • The plan• The role of Ahab’s Leg has been observer during a real project validation session• We conjecture that this is not due to the specific project, but the problem is more general• Test this conjecture in a controlled and repeatable in-lab experiment – We control/measure all the relevant variables – We change just one variable and we study the effect
  • Research questions• Ahab’s Legs are often unavoidable• Not a problem, unless they divert the attention from important aspects of the story.RQ1: What is the actual impact of Ahab’s Legs on thedistraction of stakeholders during a requirementvalidation sessions?RQ2: Is there a reliable way to reduce their impact onthe distraction?
  • Context of the experiment• Subjects: Computer science master students. – Background on software and requirement engineering – Some actual development experience• Objects: 2 software system found on the internet – MyBanking: home banking application for mobile devices, designed to replace credit cards and cash. – MyShopping: augmented reality application for mobile devices that display information on items pointed by the camera.
  • Hypothesis formulation• H0: explicitly mentioning what are the over- specified details (Ahab’s Legs) in application scenarios does not significantly reduce the distraction in a requirement validation session.• HA: explicitly mentioning what are the over- specified details (Ahab’s Legs) in application scenarios significantly reduces the distraction in a requirement validation session.
  • Variable selection• Independent variable: explicitly telling that there are details irrelevant for the discussion (Ahab’s Legs) or without such explicit mention.• Dependent variable: distraction observed during the requirement validation phase. – The stakeholder comment addresses a topic that is not part of the requirement (e.g., Ahab’s Leg) distractioni =1 – The comment requires to fix/reconsider a requirement distractioni =0 – Disrtaction = distractioni
  • Co-factors that we measure• Learning effect between the two labs• System for which requirements are validated• Subjects’ academic merit as the average of exam score• Subjects’ background measured as they attended relevant courses• Previous subjects’ experience – On requirement validation – On industrial software development
  • Experimental design Group1 Group2 Group3 Group4Lab1 MyBanking AL MyBanking No MyShopping AL MyShopping NoLab2 MyShopping No MyShopping AL MyBanking No MyBanking AL • Fill the profiling pre-questionnaire • Lab 1 – Read the description of the first application – For each of the 4 scenarios • Read a scenario • Write a comment/question for the scenario • Lab 2 – Read the description of the first application – For each of the 4 scenarios • Read a scenario • Write a comment/question for the scenario • Fill the feedback questionnaire.
  • Missing aspects?• Other strategies to control the influence of Ahab’s Leg dilemma?• Trade-off between distraction and level of abstraction?• What is the influence of subject background?
  • Conclusions
  • Questions?