This document discusses research on multi-modal scientific communication and presentations. It describes three scenarios for studying audience reception and attention allocation during presentations using eye tracking and think aloud methods. Scenario I examines live presentations across various fields, finding differences in where audience attention was focused depending on the discipline. Scenarios II and III were to be conducted in a lab setting. The conclusions propose hypotheses about how meaning is composed from multiple communication elements and how coherence is achieved in multi-modal presentations through design and signaling. The overall goal is to better understand knowledge transfer through different presentation modes.

1. How‘evil‘ is PowerPoint really? Eye-Tracking as a method in multi-modal scientificcommunicationresearchStavanger | 05/06/2009 1 Hans-Jürgen Bucher | Philipp Niemann

2. Areas of Interest 2 Hans-Jürgen Bucher | Philipp Niemann

3. Areas of Interest 3 Hans-Jürgen Bucher | Philipp Niemann Surrounding Area Multimodality Non-Linearity Presentation Speaker Audience / Moderator

4. 4 Aspectsofunderstandingmultimodal communication “Writing moves the sound world to a world of visual space, but print locks words into a position in this space. […] Because print controlled not only what words were put down to form a text but also the exact situation of the words on a page and their spatial relationship to one another, the space itself on a printed sheet took on high significance.” (Ong 2002/1988: 119, 122, 126) Sequentialorder:temporal logic Presentation: spatio-temporal logic Compositionalorder:spatiallogic Hans-Jürgen Bucher | Philipp Niemann

5. Aspects of design in presentations The spatial order / arrangement of presenter(s), screen(s) The presenter’s actions of verbal, physical and technical reference to elements of the slide The design of the PP-slides (bullet points, typography, layout, visualisations etc.) The animations of the slides and the slide transitions 5 Hans-Jürgen Bucher | Philipp Niemann

6. Research problems Problem of Compositionality: How is the meaning of a multimodal document composed out of the single elements?How are the different semiotic systems coordinated in the process of meaning making?(holistic versus additive) Problem of Reception:Are the processes of selection and meaning making controlled by attributes of the document or the recipient? How do visual salience and cues or aspects of speechinfluence the process of reception? (inductive versus deductive) 6 Hans-Jürgen Bucher | Philipp Niemann

7. Research questions How is attention allocated during multi-modal scientific presentations? How can coherence emerge in multi-modal scientific communication? Are therespecificpatternsofreception? How are communication problems solved that appear during the reception of multi-modal presentations? Do newmodesofscientificpresentationcontributetooptimizingknowledgetransfer? 7 Hans-Jürgen Bucher | Philipp Niemann

9. Think aloud

11. Humanities

12. Sciences

13. Economics Hans-Jürgen Bucher | Philipp Niemann

14. Scenario I: Setting 10 Hans-Jürgen Bucher | Philipp Niemann

15. Scenario I: Analysis 11 Hans-Jürgen Bucher | Philipp Niemann

16. Scenario I: Results (meso level) 12 Hans-Jürgen Bucher | Philipp Niemann Engineering science:

17. Scenario I: Results (meso level) 13 Hans-Jürgen Bucher | Philipp Niemann

18. Scenario I: Results (meso level) 14 Hans-Jürgen Bucher | Philipp Niemann

19. Scenario I: Results (meso level) 15 Hans-Jürgen Bucher | Philipp Niemann Law:

20. Scenario I: Analysis 16 Hans-Jürgen Bucher | Philipp Niemann

21. Scenario I: Results (microlevel) 17 Hans-Jürgen Bucher | Philipp Niemann Engineering science: Speaker Graphics

22. Scenario I: Results (microlevel) 18 Hans-Jürgen Bucher | Philipp Niemann

23. Scenario I: Results (microlevel) 19 Hans-Jürgen Bucher | Philipp Niemann Experimental economics: Speaker Graphics Text Other

24. Scenario II + III: Laboratory setting Scenario II Scenario III 20 Hans-Jürgen Bucher | Philipp Niemann : :

25. Conclusions (Hypotheses) The meaning of a presentation is compositional. Following a presentation means to organize attention simultaneously on a temporal and a spatial level. The perception of the slides is determined by their rhetorical style (and not only by visual salience). All elements of the presentation are contextualizing each other. Multimodal meaning is multiplicative and not additive. Coherence is organized by the elements of design: the actions of reference (implicit and explicit), and visual cues. Spatial contiguity is replaced by semantic contiguity. Reducing the cognitive overload of a presentation depends on the coherence management (signaling principle). 21 Hans-Jürgen Bucher | Philipp Niemann

26. Interactive Science Interdisciplinaryresearchassociation: ComputationalLinguistics (Gießen) Political Science (Gießen) Media Science (Trier) Dramatics (Hamburg) Information Science (Konstanz) Technology Assessment (Wien) Project duration: 2008-2011 Funding: Volkswagen Foundation 22 Hans-Jürgen Bucher | Philipp Niemann