7. A Glimpse of the Future
• Combining current technology trends offers
the potential to create memory augmentation
systems that radically change the way we think
about and use human memory.
• Wide range of application areas including:
- Behaviour Change
- Learning
- Supporting Failing Memories
- Selective Recall
- Recall Based Advertising
8. Examples
• Learn names of new
colleagues
• Learn a new
language
• Remember
important events
17. Eco-systems for Enhanced Human Memory
Storage and
Processing
Environmental
Sensors
User Sensors
Application Data
& Cloud Services Personal
Devices
Public
Infrastructure
Experience Capture
Presentation
and Review
18. Summary
• In the future mobile and ubiquitous systems
are going to be used to enhance human
cognition.
• Building combined wearable, mobile and
infrastructure support for memory
augmentation offers new opportunities and
raises many new challenges.
19. Sarah Clinch
School of Computing and Communications
InfoLab21, Lancaster University
s.clinch@comp.lancs.ac.uk
The research leading to these results has received funding from the
European Union Seventh Framework Programme (FP7/2007-2013) under
grant agreement number 612933.
http://recall-fet.eu/
Editor's Notes
Why are we capturing information?
RECALL specifically aims to support episodic and autobiographical memory, i.e. memory of actual experiences.
How are we going to use this captured information?
RECALL is primarily aiming to acquire cues that can be played back in order to improve memory recall.
near-continuous collection of memory cues has become possible
Microsoft’s SenseCam,
social networks
interaction logs.
Challenge – Information sources
Images and Video
Personal vs Infrastructure Acquisition
Quality Assessment
Non-visual data
Mobile Phone
Email & Calendar Info
near-continuous collection of memory cues has become possible
advances in data storage and processing
enables widespread mining of stored cues both in terms of cues collected by an individual and in terms of complex networks of related cues contributed by others.
Challenge – acquisition & access
Joint Memory
How can I receive your memory stream?
How to distribute infrastructure memory?
Sharing Memory
Who owns memory?
How to control memory distribution
the presence of ubiquitous displays
both in the environment and via personal devices (phones, watches, glass)
provides many new opportunities for displaying memory cues to trigger recall.
Challenge – context aware cueing
Training Recall Through Cuing (Replay)
What to replay? Extracting key episodes in recorded events
When/Where to replay? Ambient / wearable
Re-think and re-define the notion of memory augmentation
Using capture technology and information retrieval to nurture the human mind
Fundamentally improve physiological parameters, not just clever prosthetics!
Explicit capture:
Smartphone camera roll, GoPro video stream
Location checkin, calendar entry, social media post
Implicit capture:
SenseCam / NC / MeCam photo/video stream
WiFi/BT presence detection / GPS trace
Proximity detection / conversation detection
Wearable health monitoring data
Stable, carefully positioned sensors
Typically larger field of view (i.e. more memory cues)
Reuse of existing infrastructure
Removes constraints of mobility (e.g. size/power consumption) and wearability (e.g. aesthetics)
Moves maintenance burden away from users
2.75 day study including 21 researchers
Stayed in an instrumented hostel environment
Wore/carried a variety of mobile capture devices
Schedule incorporating a wide range of tasks
e.g. cooking, hiking, dining, workshop discussions
42,959 images (34.9 Gigabytes)
248.15 hours of video (245.2 Gigabytes)
Plus indoor and outdoor location traces
Re-think and re-define the notion of memory augmentation
Using capture technology and information retrieval to nurture the human mind
Fundamentally improve physiological parameters, not just clever prosthetics!
We can do this …
the wrong way: users carry ever increasing amounts of technology and infrastructure owners collect and act on ever-richer user profiles
the right way: users control how and when infrastructure is leveraged to augment cognition without revealing personal information
Building combined mobile and infrastructure support for memory augmentation raises many new challenges.
Experience Provenance
how do users ensure the provenance of the data they store as memories?
Memory Protection
How can we develop mechanisms to enforce the wide variety of policies desired by individuals to exercise control over access to their memories? (Technical/social/legal)
Memory Manipulation
How can a user tell if their memories are being manipulated?
Privacy of Bystanders
How to protect bystanders while still allowing substantial data collection for human memory augmentation?