This document studied the effects of different tasks and phases on nonverbal communication in computer-mediated interactions. It found: 1) Rates of nonverbal behaviors like gaze and gestures varied based on whether the task required verbal coordination or spatial problem-solving. 2) Phases within tasks also showed differences, with planning emphasizing gaze and communicative gestures more than execution. 3) The results confirmed that the impacts of mediated communication depend on the task, supporting prior research.

1. Effects of tasks and phases on
nonverbal communication
Liv Lefebvre - Alexandre Pauchet - Laurence Perron
France Telecom R&D
University of South Britany, France

2. Mediated communications
2

3. Theorical basis
3

4. Theorical basis
 Computer Mediated Communications studies:
– Face to face
– Audio-visual
– Audio alone
 Results : consequence of absence of visual cues :
– Management of communication turn is less fluent
– Reference to the shared environment must be more explicit
– Less mutual comprehension
– Negotiation and consensus more difficult to find, less solutions
 BUT Whittaker's hypothesis (2003) : effects of mediated
communication are task dependent
– Resolution problem tasks: few differences
– Task that require access to interpersonal information: larger
differences
4

5. The experiment
5

6. Task comparison
Narrative co-conceptionPuzzle completion task
"create the most beautiful and "complete the puzzle as quickly
Instructions : coherent story possible with as possible with your partner
your partner" "
Participants: 18 dyads 16 dyads
6

7. Task comparison: narrative co-conception task
7

8. Task comparison: puzzle completion task
8

9. Task comparison
Narrative co-conceptionPuzzle completion task
Nature of the task : verbal spatial
Nature of the solution : not known known
9

10. Technical device
10

11. Measurement
 Nonverbal production:
– Gaze
11

12. Measurement
 Nonverbal production:
– Gaze
– Communicative gestures: serve to complete and module
discourse (Ekman & Friesen, 1969 ; Goldin-Meadow, 1999 ;
McNeill, 1992)
– Metaphorics
– Beats
metaphoric beat 12

13. Measurement
 Nonverbal production:
– Non communicative gestures (adaptors):
– are gestures produced in case of personal
disorganization (Ekman and Friesen, 1969)
– they don't have any relation with social interaction
(Argentin, 1984 ; Masse, 2000)
adaptators 13

14. Measurement
 Nonverbal production:
– Gaze
– Communicative gestures: serve to complete and module
discourse (Ekman & Friesen, 1969 ; Goldin-Meadow, 1999 ;
McNeill, 1992)
– Metaphorics
– Beats
– Non communicative gestures (adaptors):
– are gestures produced in case of personal disorganization
(Ekman and Friesen, 1969)
– they don't have any relation with social interaction
(Argentin, 1984 ; Masse, 2000)
 Rate of nonverbal production = ( time spend to product /
session's time ) * 100
14

15. Results
15

16. Results: comparison between two tasks
16
14
Rates of nonverbal production
12
Non communicative
10
gestures
8
Communicative
gestures
6 Gaze
4
2
0
puzzle completion narrative co-
task conception task
16

17. Results: comparison between two tasks
Puzzle completion Narrative co-
task conception task
Gaze 16 % 44 %
Communicative gestures 6% 7%
Non communicative
gestures 79 % 49 %
 Patterns of nonverbal production are task dependent
 Can see his partner is not necessary in puzzle completion task…
 …but there isn't the case in narrative co-conception task
 Confirmation of the Whitaker's hypothesis
17

18. Results: comparison between phases in the
puzzle completion task
 Comparison between phases in the puzzle completion task
– Planning phase
– Execution phase
18

19. Results: comparison between phases in the
puzzle completion task
16
Rate of nonverbal production
14
12
Non communicative
10 gestures
8 Communicative
gestures
6 Gaze
4
2
0
Planning
Planification Execution
Execution
phase
phase phase
phase
19

20. Results: comparison between phases in the
puzzle completion task
 Big difference in non verbal production
Planning phase Execution phase
Gaze 53 % 7%
Communicative gestures 42% 4%
Non communicative
gestures 5% 90%
20

21. Results: comparison between phases in the
narrative co-conception task
 21 dyads interact via computers
– Same task
– Same instructions
 Differences between phases are
less clear:
– Review of items
– Narrative phase
– Interaction phase
21

22. Results: comparison between phases in the
narrative co-conception task
40
35
Rates of nonverbal production
30
Non communicative
25 gestures
Communicative gestures
20
15 Gaze
10
5
0
Review of Narrative Interaction
items phase phase
22

23. Results: comparison between phases in the
narrative co-conception task
Review of items Narrative phase Interaction phase
Gaze 34% 30% 46%
Communicative
gestures 2% 13% 9%
Non
communicative
gestures 64% 57% 45%
23

24. Results summary
 Production of gaze:
– In puzzle completion task in planning phase: 53 %
– In narrative co-conception task: 44 %
– In interaction phase: 46 %
 Production of communicative gestures:
– In puzzle completion task in planning phase: 42 %
 Production of non communicative gestures:
– In puzzle completion task: 79 %
– In execution phase: 90 %
24

25. Conclusion
25

26. Conclusion
 Whittaker's hypothesis (2003) is verified:
– During the narrative co-conception: more gaze and communicative
gestures
– During puzzle completion: more non communicative gestures
 In addition: differences between phases in interactions:
– Planification phase: more gaze and communicative gestures
– Execution phase: more non communicative gestures
 Applications for design of new systems of mediated communications
– Importance of visual access:
– During tasks that require access to interpersonal information
– When users have need to negotiate the task planification
– Not necessary to have a visual access:
– During resolution problem tasks
– When users have already plan the task and they execute their
plans
26

27. thank you
contact: liv.lefebvre@gmail.com
Whittaker, S. (2003). Theories and Methods in Mediated Communication. In A. C. Graesser &
M. A. Gernsbacher &S. R. Goldman (Eds.), Handbook of Discourse Processes. Mahwah: NJ:
LEA.

28. 28

29. 29

30. Experimental manipulation
 4 experimental conditions:
– At a distance with the same view on the material
– At a distance with opposite view on the material
– Collocated situation side-by-side
– Collocated situation face-to-face
30

31. Experimental manipulation
 4 experimental conditions:
– At a distance with the same view on the material
– At a distance with opposite view on the material
– Collocated situation side-by-side
– Collocated situation face-to-face
31

32. Experimental manipulation
 4 experimental conditions:
– A a distance with the same view on the material
– A a distance with opposite view on the material
– Collocated situation side-by-side
– Collocated situation face-to-face
32

33. Experimental manipulation
 4 experimental conditions:
– A a distance with the same view on the material
– A a distance with opposite view on the material
– Collocated situation side-by-side
– Collocated situation face-to-face
33

34. Results by conditions
Distance
16
14
Rates of nonverbal production
12
Non communicative
10
gestures
8
Communicative gestures
6 Gaze
4
2
0
puzzle completion narrative co-
conception
34

35. Results by conditions
collocated
7
6
Rates of nonverbal production
5
Non communicative
gestures
4
Communicative
gestures
3
Gaze
2
1
0
puzzle completion narrative co-
conception
35

36. Results by conditions
face-to-face
14
12
Rates of nonverbal production
10
Non communicative
gestures
8
Communicative
gestures
6
Gaze
4
2
0
puzzle completion narrative co-
conception
36

37. Results by conditions
Side-by-side
14
Rates of nonverbal production
12
10
Non communicative
gestures
8
Communicative
gestures
6
Gaze
4
2
0
puzzle completion narrative co-
conception
37