Search

Nzps 2009 Aviation Reasoning

Loading...

Flash Player 9 (or above) is needed to view presentations.
We have detected that you do not have it on your computer. To install it, go here.

0 comments

Post a comment

    Post a comment
    Embed Video
    Edit your comment Cancel

    Notes on slide 1

    Abstract When pilots become lost they are often faced with a situation that is analogous to Wason's (1960) famous rule discovery task. Wason showed that people often attempt to tackle the task using suboptimal strategies that make it difficult to discover the underlying rule. We examined the possibility that student pilots’ domain specific knowledge of navigation would protect against the use of these suboptimal strategies. In a series of five navigation-oriented studies we found that not only did pilots and non-pilot students exhibit classic confirmation bias but that even quite explicit instructions to use optimal strategies did not improve performance. Interestingly we found evidence to suggest that domain-specific experience can protect against suboptimal strategy use – a sample of orienteers performed significantly above chance in the same navigation tasks. The theoretical and practical significance of these findings is discussed.

    Roughly speaking, confirmation bias is tendency to seek out or interpret new information in a way that confirms one's preconceptions, but avoid information that contradicts them. Confirmation bias means finding what one is looking for – the idea is often considered to be a, possibly unconscious, motivational bias where one has a goal of bolstering a preferred hypothesis. But most research on confirmation bias shows that CB, if it occurs at all, is more likely to be a cognitive bias – a byproduct of our normal way of reasoning about the world.

    Air New Zealand flight TE901, 28 th November 1979, 257 people died. “ Those on the flight deck interpreted features of the topography ahead in such a way as to confirm their mindset, not challenge it. The flat slopes of Ross Island and Beaufort Island were misinterpreted as features associated with McMurdo Sound instead of Lewis Bay at the foot of Erebus.” (p. 21 of McGregor, A. (March, 2006). Accidents, failures, mistakes, and leaky buildings . A Paper presented at the National IPENZ Conference, Wellington, New Zealand.

    The term confirmation bias (CB) is used to describe a lot of different ideas (Nickerson, 1998) The kind of CB we are interested in cognitive not motivational in nature.

    Wason referred to the tendency for Ps to ask ‘yes’ questions in order to confirm their hypothesis as enumerative induction (the use of confirming evidence only). But Wetherick (and later Klayman & Ha) pointed out that eliciting a ‘no’ answer could also be consistent with a confirmatory strategy (and a ‘yes’ answer might be consistent with a disconfirmatory strategy). One might try to confirm H1 by using a triple NOT consistent with H1 and seeking a no response. One might try to disconfirm H1 by using seeking a positive response to an inconsistent (with H1) triple. Thus, one needs to distinguish between a positive test strategy (which can lead to disconfirmation/falsification) and an attempt to confirm one’s favoured hypothesis. In the RDT these two phenomena are highly correlated but in other situations they need not be. Poltiek (1996) showed that participants find it difficult to make sense of the instruction to use a falsificatory strategy (try to prove false) one’s best guess. She argues that this is an incoherent idea – one’s best guess is, by definition, what one thinks is likely to be the case; thus, one can be expected to find the correct rule by attempting to disprove one’s best guess about that rule. Participants use both positive and negative testing to try and gather evidence for their best guess. Evans’ (1989, 2006) heuristic/analytic framework shows that confirmation bias (a tendency to think one’s hypothesis is correct because of the strategies used to evaluate it) occurs, not because we have a confirmation bias tendency, but rather because of the operation of three principles: singularity (people typically can only focus on one hypothesis), relevance (the hypothesis is the most plausible or probable in the context), and satisficing (the hypothesis is evaluated in terms of current goals and accepted if satisfactory).

    When PTS goes wrong … When H is embedded in T … PTS cannot distinguish R T from R H Only a negative testing strategy (NTS) can In context of the Rule Discovery Task PTS: {14, 16, 18} or {8, 10, 12} All triples consistent with H are also consistent with T NTS: {3, 5, 7} or {9, 10, 11} or {20, 30, 40} By selecting a triple that one does NOT expect to be consistent with R H and getting a YES answer one discovers that is R H false

    Check the time elapsed since the last position fix and estimate the distance covered in that time. On the chart draw a line of position [LOP] , across the track (the original or an intercept), at the estimated distance from the last fix. The line should extend about 1 nm either side of track, for each 5 minutes flown since the fix. i.e. if it is 30 minutes since your last positive fix then the line will extend roughly 6 nm either side. Then draw a rough circle with the LOP as the diameter (refer diagram below) and your most probable position [MPP] is somewhere within that circle of uncertainty. Find the most prominent features on the map within the circle and then try to locate them on the ground. The 1 nm per 5 minutes is based on ground speeds around 50 or 60 knots, if ground speeds are around 100 knots then make it 2 nm per 5 minutes. 'Most probable' means maybe an 80% chance. "Read from ground to map!" Normally in flight the navigator should be continually identifying features on the map and waiting for the next one to come up on track, within an estimated time. When uncertain of position the procedure is reversed, look for two or more large features on the ground and then identify features on the chart that are in the same juxtaposition. Prominent line features are best although, quite often, a spot feature is easily identified – for example the names of grazing or farming properties are shown on the charts and their owners, particularly those with an airstrip, often paint the name on a roof, in large letters. If you see a prominent line feature, then fly along it until you can derive a fix from an intersect or a verifiable landmark.

    We carried out 5 separate studies in which participants were asked to imagine they were lost, although they were probably located within a red circle drawn on the map (the circle represented their theorised location). Participants were told they could see three distinctive features from where they were truly located and to select one feature that was most useful to determine whether they were in the circled location. They were told there could be dire consequences of wrongly believing they were in the circle, if in reality they were not.

    Diagnosticity = how good a test is at distinguishing between two options.

    Why are orienteers better than pilots? Why were pilots no better than non-experts? Evidence mixed on power of expertise to improve performance on RDT and related tasks Gorman et al. clergy better than scientists on RDT But expert software engineers better than novices (Teasley et al. 1994) – although still strong evidence for positivity bias and insufficient testing by experts. Theorists argue that people find it very difficult to naturally consider multiple hypotheses Evans’ principle of singularity But this does increase performance (dual goal studies). DG = DAX and MED examples. Allows positive testing to be used successfully Are orienteers inclined to consider dual goals? Are they used to being put in situations where they are ‘tricked’ to think they are located where they are not? Could this drive a natural inclination to dual goal use?

    What is the best solution for training pilots? Encouraging negative testing is difficult Ps often don’t even think it (a falsificatory strategy) is possible (Poltiek, 1996) Klayman & Ha (1987) suggest we have a positivity bias because PTS works well in so many situations Difficult to consider more than one hypothesis (Principle of singularity) Many lost situations are perfectly amenable to a single hypothesis, positive testing strategy Therefore not a lot of demand for dealing with ‘embedded situations’ Compare that to orienteering where embedded situations may well be common

    Use highly diagnostic landmarks wherever possible (e.g., hills with trees on is less useful than farm homestead with the name ‘Old McDonald’ on the top) Use multiple landmarks to increase chance of use of a unique landmark Individuate landmarks as much as possible to increase diagnosticity (a map may symbolise two different looking landmarks in the same way – e.g., two settlements. Thus, a pilot can use their own experience to further individuate landmarks or, if this is not possible, try to use landmarks that ARE individuated on a map – e.g., odd shaped roads or rivers).

    Favorites, Groups & Events

    Nzps 2009 Aviation Reasoning - Presentation Transcript

    1. BIASES IN REASONING IN AVIATION NAVIGATION Stephen Hill 1 and Andrew Gilbey 2 1. School of Psychology 2. School of Aviation
    2. Confirmation Bias
      • “ A tendency to seek out or interpret information in a way that confirms one’s preconceptions and avoid information that contradicts them”
    3. Confirmation Bias and the Erebus Disaster True Course PerceivedCourse DC 10, ZK NZP
    4. Confirmation Bias and the Erebus Disaster “ Those on the flight deck interpreted features of the topography ahead in such a way as to confirm their mindset, not challenge it. The flat slopes of Ross Island and Beaufort Island were misinterpreted as features associated with McMurdo Sound instead of Lewis Bay at the foot of Erebus.” (McGregor, 2006, p. 21).
    5. Confirmation Bias (CB)
      • A many splendoured thing
        • ‘ Motivated’ CB versus ‘Cognitive’ CB
      • Term used ‘rather loosely’ in psychology and related disciplines
      • Most famous and popular CB tasks
        • Peter Wason’s 4 card task
        • Wason’s Rule Discovery Task
    6. The Rule Discovery Task
      • Wason (1960)
        • Ps required to discover experimenter’s rule ( R T ) (e.g., ascending numbers) by asking ‘yes/no’ questions
        • Given a ‘seed’ triple 2-4-6
        • Ps can propose as many triples as they like in order to work out R T
      • Findings
        • Ps exhibit a high degree of confidence in incorrect hypotheses (~20% success rate, 75% eventually get it right)
        • Ps typically guess (their R H ) the rule is ‘3 ascending even numbers’
        • Ps adopt a positive testing strategy (PTS) = trying out triples for which the answer is expected to be ‘yes’ given R H
    7. ‘ Confirmation Bias’ and the Positive Testing Strategy (PTS)
      • Problems with Confirmation Bias
        • Wason conflated use of a PTS with an attempt to confirm one’s favoured hypothesis (e.g., Wetherick, 1962)
        • Also unclear whether it makes sense to insist that people try and ‘falsify their best guess’ (Poltiek, 1996)
        • Not a ‘principle of reasoning’ (e.g., Evans, 1989, 2006)
        • Not nearly as common as once thought (e.g., McKenzie, 2006)
      • CB is best seen as a suboptimal outcome rather than a suboptimal behavioural strategy
        • Focus has moved to the choice of test strategies used by Ps
          • PTS is a useful strategy in many contexts (Klayman & Ha, 1987, 1989)
    8. Possible Relationships between H and T H T H H H T T T Disjoint Overlapping Embedded Surrounding U U U U PTS is only really problematic when H is embedded in T
    9. Getting Lost
      • Unlike Erebus and similar events, most lost pilots successfully get ‘un-lost’
        • Suggests pilots might be pretty good at testing location hypotheses
      • Pilots are instructed to follow ‘General Aviation Lost Procedures’*
        • Orbit a landmark
        • Draw a circle of uncertainty (COU) on map
        • Look for a distinctive landmark (e.g., farm name on roof) on the ground and try and find it in COU
        • *NB: only if a pilot realises they are lost of course.
    10. Confirmation Bias and Navigation
      • Aim: Construct a navigation equivalent of the RDT based on a ‘toy version’ of general aviation lost procedures
      • Explore whether …
        • Pilots are better than the average person at ‘resisting confirmation bias’ in a navigation situation.
        • Find out what kinds of strategies are used to reason about location.
    11. The Location Discovery Task
      • Mapping the RDT on to the LDT …
        • R T -> where you really are
        • R H -> where you think you are (COU)
        • Triples -> landmarks
      • All relationships between R T and R H can be explored but we were most interested in the (difficult) embedded relationship
    12. The Location Discovery Task
      • 5 studies
        • Level of expertise varied (students, pilots, orienteers)
        • Information given about CB (control, intervention)
      • Ps asked to imagine being lost
      • Given a ‘circle of uncertainty’ (-> R H for most people)
      • Asked to select 1 of 3 visible landmarks that is best for working out one’s location
        • * We also asked people to provide reasons for their choices
      • There is a picnic area approx 20m from where you are standing and a stream crosses under the road
      • In the distance to the east there are high bush covered ranges
      3. There is a junction and to your left there is an unsealed road (light brown) leading off to the west + + - Motorcycle Accident Scenario
    14. Chance level * * ** * * Significantly worse than chance ** Significantly better than chance Mean Disconfirmatory Responses (out of 3)
    15. Summary of findings for all studies t (28) = -2.20, p < .05 .71 1 2 13 13 29 example t (16) = -1.57, n.s. .69 0 3 6 8 17 no instruction t (45) = -2.73, p < .05 .70 1 5 19 21 46 cognitive psychology students 5. example and no instruction t (17) = -.89, n.s. .78 2 2 4 10 18 student pilots 4. example t (20) = 6.48, p < .001 2.00 5 11 5 0 21 orienteers 3. no instruction t (35) = -4.78, p < .001 .56 0 1 18 17 36 cognitive psychology students 2. lecture on confirmation bias t (65) = -4.90, p < .001 .58 0 8 22 36 66 student pilots 1. no instruction 3 2 1 0 Difference from chance M disconfirmatory choices n disconfirmatory choices (0 to 3) Participants Study
    16. Peformance within Scenarios
      • Performance in scenarios varied widely
          • Sea Scenario 8.0% correct on average
          • Motorcycle Scenario 46.5%
          • Flying Scenario 24.1%
      • Despite this performance rankings were very similar across scenarios
          • Orienteers
          • Pilots (intervention)
          • Psychology students (intervention)
          • Pilots (non-intervention)* - except for Sea where Pilots were 5 th
          • Psychology students (non-intervention)** - except for Sea where Psychology students were 4 th
    17. Performance Across Scenarios Scenario (in order of difficulty) Proportion Correct
    18. Peformance Across Scenarios
      • Why did performance vary so dramatically across scenarios?
        • It appears many Ps might not be using the map at all!
          • Landmarks may have been selected based on how uncommon (diagnostic) Ps believed them to be
    19. Railway line with branch line leading off east  Large pine plantation either side of road and railway 24.1% Flying Picnic area with stream running under road  Picnic area with stream running under road 46.5% Motorcycle Small aircraft landing behind town  High bush clad peak behind town 8% Sea Most common choice Correct landmark Correct Scenario
    20. Implications
      • Why are orienteers better than pilots?
      • Why were pilots no better than non-experts?
        • Evidence mixed on power of expertise to improve performance on RDT and related tasks
        • Theorists argue that people find it very difficult to naturally consider multiple hypotheses
          • But dual goal studies show it helps if you can
        • Are orienteers inclined to consider dual goals?
          • Are they used to being put in situations where they are ‘tricked’ to think they are located where they are not?
          • Could this drive a natural inclination to dual goal use?
    21. Implications
      • What is the best solution for training pilots?
        • Encouraging negative testing is difficult
          • Klayman & Ha (1987) suggest we have a positivity bias because PTS works well in so many situations
        • Difficult to consider more than one hypothesis (Principle of singularity)
        • Many lost situations are perfectly amenable to a single hypothesis, positive testing strategy
          • Therefore not a lot of demand for dealing with ‘embedded situations’
    22. Implications
      • What is the best solution for training pilots?
        • Perhaps change the situation to one where a single hypothesis combined with PTS is most likely to work?
          • = turn situation (relationship between T and H ) into a disjoint , or at worst, overlapping one
        • How can we do this?
          • Use highly diagnostic landmarks wherever possible
          • Use multiple landmarks to increase chance of use of a unique landmark
          • Individuate landmarks as much as possible to increase diagnosticity
        • BUT these strategies won’t guarantee avoiding CB so also …
          • Teach them about confirmation bias?
          • Perhaps encourage them to do some orienteering …?
    SlideShare Zeitgeist 2009

    + srhillsrhill Nominate

    custom

    114 views, 0 favs, 0 embeds more stats

    Stephen Hill and Andrew Gilbey
    Confirmation Bias in more

    More info about this document

    © All Rights Reserved

    Go to text version

    • Total Views 114
      • 114 on SlideShare
      • 0 from embeds
    • Comments 0
    • Favorites 0
    • Downloads 4
    Most viewed embeds

    more

    All embeds

    less

    Flagged as inappropriate Flag as inappropriate
    Flag as inappropriate

    Select your reason for flagging this presentation as inappropriate. If needed, use the feedback form to let us know more details.

    Cancel
    File a copyright complaint
    Having problems? Go to our helpdesk?

    Categories

    Tags

    -->
    Search
    RSS Feed
    What's new?

    © 2009 SlideShare Inc. All Rights Reserved