Presentation given at AVI 2018, International Working Conference on Advanced Visual Interfaces, Grosseto, Italy, May 2018 ABSTRACT: Audio recordings and the corresponding transcripts are often used as prosthetic memory (PM) after meetings and lectures. While current research is mainly developing novel features for prosthetic memory, less is known on how and why audio recordings and transcripts are used. We investigate how users interact with audio and transcripts as prosthetic memory, whether interaction strategies change over time, and analyse potential differences in accuracy and efficiency. In contrast to the subjective user perception, our results show that audio recordings and transcripts are equally efficient, but that transcripts are generally preferred due to their easily accessible contextual information. We further identified that prosthetic memory is not only used as a recall aid but frequently also consulted for verifying information that has been recalled from organic memory (OM). Our findings are summarised in a number of design implications for prosthetic memory solutions. Paper: https://www.academia.edu/36500351/Designing_Prosthetic_Memory_Audio_or_Transcript_That_is_the_Question

1. 2 December 2005
Designing Prosthetic Memory:
Audio or Transcript, That is the Question
Sandra Trullemans, Payam Ebrahimi and Beat Signer
Web & Information Systems Engineering Lab
Department of Computer Science
Vrije Universiteit Brussel
WEB & INFORMATION
SYSTEMS ENGINEERING

2. Payam Ebrahimi - Department of Computer Science - pebrahim@vub.be 2May 30, 2018

3. Payam Ebrahimi - Department of Computer Science - pebrahim@vub.be 3May 30, 2018
Cross-Indexed Prosthetic Memory
ChittyChatty
Kalnikaité et al. 2007
Ferret browser
Wellner et al. 2004
Use Digital Prosthetic Memory
a Kalnikaitė
eld University
, Sheffield, S1 4DP, UK.
e@sheffield.ac.uk
Steve Whittaker
Sheffield University
211 Portobello St, Sheffield, S1 4DP, UK.
s.whittaker@sheffield.ac.uk
orable and important moments, as
f information. The last few years
on of digital devices intended to
ory (PM), to help users recall
ons and retrieve personal
theless have little systematic
nd why people might use such
heir own organic memory (OM).
, it may be more efficient than
om a complex PM device. We
udy which investigates when and
d OM. We found that PM use
tion of the quality of their OM, as
rties. In particular, we found that
and Efficiency, preferring rapid
curate information over laborious
ation. We discuss the implications
e PM design and theory. Rather
e PM designs need to focus on
work in synergy.
mory, Digital Memory, Speech
al, Notes, Remembering.
words
ces and presentation (e.g., HCI):
In his oft-cited vision of the future, Vannevar Bush [2]
proposed Memex, a tool designed to help users remember
and index information they have previously encountered. In
the last 15 years this vision has become a reality.
Reductions in the cost of digital storage and the emergence
of sophisticated recording technologies have led to the
development of many different PM systems. One type of
Fig 1. ChittyChatty Graphical User InterfaceLivescribe 3

4. Payam Ebrahimi - Department of Computer Science - pebrahim@vub.be 4May 30, 2018
RQ1: Is there a difference in the way
users interact with notes, transcripts and
audio recordings and do the interaction
strategies change when time elapses?

5. Payam Ebrahimi - Department of Computer Science - pebrahim@vub.be 5May 30, 2018
RQ2: Is there a difference in accuracy and
efficiency between audio and transcript PM?

6. Payam Ebrahimi - Department of Computer Science - pebrahim@vub.be 6May 30, 2018
RQ3: Do factors such as confidence
influence the way users interact with PM?

7. Payam Ebrahimi - Department of Computer Science - pebrahim@vub.be 7May 30, 2018
Case 1 - Audio Only

8. Payam Ebrahimi - Department of Computer Science - pebrahim@vub.be 8May 30, 2018
Case 2 - Transcript Only

9. Payam Ebrahimi - Department of Computer Science - pebrahim@vub.be 9May 30, 2018
Procedure

10. Payam Ebrahimi - Department of Computer Science - pebrahim@vub.be 10May 30, 2018
Procedure - Phases
Phase 1
Phase 2
Open
Interview

11. Payam Ebrahimi - Department of Computer Science - pebrahim@vub.be 11May 30, 2018
RQ1: Is there a difference in the way
users interact with notes, transcripts and
audio recordings and do the interaction
strategies change when time elapses?

12. Payam Ebrahimi - Department of Computer Science - pebrahim@vub.be 12May 30, 2018
Overall Usage
0
20
40
60
80
100
0 7 30
LikelihoodofUsage(%)
Time Interval (days)
OM Notes Audio
0
20
40
60
80
100
0 7 30
LikelihoodofUsage(%)
Time Interval (days)
OM Notes Transcript
Audio Only Transcript Only

13. Payam Ebrahimi - Department of Computer Science - pebrahim@vub.be 13May 30, 2018
Overall Usage
0
20
40
60
80
100
0 7 30
LikelihoodofUsage(%)
Time Interval (days)
OM Notes Audio Transcript
Audio and Transcript Combined

14. Payam Ebrahimi - Department of Computer Science - pebrahim@vub.be 14May 30, 2018
0
20
40
60
80
100
0 7 30 0 7 30
Time Interval (days) Time Interval (days)
LikelihoodofUsage(%)
Transcript
Navigation
Skimming
Audio
Different Ways of Interacting with PM
Audio Only Transcript Only

15. Payam Ebrahimi - Department of Computer Science - pebrahim@vub.be 15May 30, 2018
Different Purposes of PM Use
0
20
40
60
80
100
0 7 30 0 7 30
Time Interval (days) Time Interval (days)
LikelihoodofUsage(%)
Transcript
Recall
Verifying
AudioAudio Only Transcript Only

16. Payam Ebrahimi - Department of Computer Science - pebrahim@vub.be 16May 30, 2018
RQ2: Is there a difference in accuracy and
efficiency between audio and transcript PM?

17. Payam Ebrahimi - Department of Computer Science - pebrahim@vub.be 17May 30, 2018
Accuracy
 No significant difference between audio and transcript in
accuracy
 Accuracy does not change over time intervals
 Choice of interaction has no impact on accuracy

18. Payam Ebrahimi - Department of Computer Science - pebrahim@vub.be 18May 30, 2018
Efficiency
0
20
40
60
80
100
120
140
0 7 30
Efficiency(s)
Time Interval (days)
Transcript Navigating Transcript Skimming
Audio Navigating Audio Skimming

19. Payam Ebrahimi - Department of Computer Science - pebrahim@vub.be 19May 30, 2018
RQ3: Do factors such as confidence
influence the way users interact with PM?

20. Payam Ebrahimi - Department of Computer Science - pebrahim@vub.be 20May 30, 2018
Confidence Influence on Interactions

21. Payam Ebrahimi - Department of Computer Science - pebrahim@vub.be 21May 30, 2018
Design Implications
Co-indexing PMs Efficiency and Accuracy Recall and Verification

22. Payam Ebrahimi - Department of Computer Science - pebrahim@vub.be 22May 30, 2018
Conclusion
 Investigated the overall usage of audio and transcript PM
 Audio PM is not less efficient than transcript PM
 Three design implications
 Designing for individual or combined audio and transcript
 PM is not only chosen based on efficiency and accuracy
 Designing for recall and verification