This document discusses lifelogging, which refers to the process of digitally storing data about all life experiences for future use. It envisions a "digital self" archive that captures a person's total experiences through sensors and creates a record that grows over time. Several challenges around organizing, searching, and analyzing such large datasets are discussed. The progress made in lifelogging research from early concepts to current technologies is reviewed. Potential future opportunities and issues around privacy, data access, and long-term preservation are also examined.

1. LIFELOGGING
A LONG-TERM DATA ANALYTICS CHALLENGE
Cathal Gurrin
cathal@gmail.com & @cathal
Dublin City University, Ireland
University ofTromsø, Norway
06 November 2015 - DBTA Workshop - Lugano

2. We generate huge amounts of personal data.
The next step is lifelogging. In an era of
Lifelogging, you will log all life experience into
your digital self.

3. DIGITAL SELF
An rich archive of data
about the individual that
captures the totality of life
experience in a private,
non-forgetting archive.
An lifelog archive that
should provide ubiquitous
support.
Digital Self Lifelog

4. Why should we care?
This is happening now with devices on the market already.

5. Dr Cathal Gurrin
Faculty: DCU & UiT
Scientist: Insight Centre for Data
Analytics
Lifelogger: It is what I do…
A visual lifelog for 9+ years
70+ academic publications and one
book in the area.
Curiosity-driven scientist. I started
this work without knowing where it
would lead to.

6. INSIGHT CENTRE FOR DATA ANALYTICS
20/10/14 Slide 5
Healthier, safer and more productive world using
data analytics.

7. AGENDA
What is lifelogging?
Vision for lifelogging
How to make lifelog applications
Progress made thus far
Future expectations

8. WHAT
WHAT -VISION - HOW - PROGRESS - FUTURE

9. Lifelogging refers to the process of storing data concerning
the totality of life experience for future use…
An data-driven extension of our memory and our cognitive
abilities. It gets more valuable as it gets bigger.
cars (legs)
glasses (eyes)
hearing aids (ears)
pacemakers (hearts)
lifelog (memory)

10. We will soon be able to capture a digital trace of the totality
of life experience in digital archives

11. Data such as this…

12. 10 Years
of location
log, with
millions of
GPS points
2 Years
heartbeats,
with GSR and
activity.
Communicati
ons
1 Year
of computer
interactions
(mouse,
keyboard,
email, WWW)
9 Years
of lifelog,
since 2006
16
Million
images of
what I see for
9+ years
Grows at about 1TB per year. You can’t retrospectively gather and you
don’t know what will be important later.
Secure
Personal
Lifelog
1000+
Docs
all I read and
write and
listen to
My Digital Self (2006 - now)

13. Activities
&
Health

14. Everything I see

15. Everything I read
or write on computer
or digital paper
Removed for
privacy

16. VISION
WHAT - VISION - HOW - PROGRESS - FUTURE
From the point of view of the individual

17. Digital Self
Life Experience
A digital lifelog of all life experience…
activities, experiences, thoughts,
emotions… huge volumes of data that
last a lifetime and beyond.

18. Focus on supporting knowledge
acquisition and learning in the
early years.
1. Knowledge Support
From education to the workplace,
providing information and insights
to assist productivity and fitness.
2. Productivity
Into old age, providing support for
cognition and health to maintain
independence and activity.
3. Health
CHILD
ADULT
ELDERLY
The lifelog will be a permanent companion assisting you
throughout life. Constantly growing in size.

19. Quantified Self
Enhanced Access to
Knowledge
Learning Support
Performance
Enhancement
Personal Insights
Data-driven Health
Population-wide
Analytics
Healthcare
Enhancement
Enhancing Human
Memory
Assistive Technologies
Synergy, not
Substitution
Assisted
Memory
Better Interactions
Rich Sharing and
Reminiscing
Data Partners/Carers
Social
Enhancement
Why?To provide knowledge to empower
Quantified-Self Memory

20. Quantified-Self
Analysis
Self-discovery
Reflect
Calendar integration
& Context
Remind
Sousveillance.
Protection of me
and bystanders
Protect
Find an item from
the digital self
Validate a memory
Contextual support
Search
Find & share items
Social applications
Reminisce
Digital Agents
acting on our
behalf, during life
and after
Represent
My interest is in memory applications. So we can identify many
ways that a lifelog / digital self can assist human memory.

21. Search
www.loggerman.org

22. Quantified-Self
Analysis
Self-discovery
Reflect Visual Reflection Software - EyeAware

23. Sousveillance.
Protection of me
and bystanders
Protect

24. HOW
WHAT -VISION - HOW - PROGRESS - FUTURE

25. We experience
the world through our
senses… add in our
thoughts and emotions
Considering humans…

26. Ideally we need a sensor that can record everything you see,
hear and experience…

27. In 2015, lifelogging
can generate
thousands of
images per day,
hours of audio and
tens of thousands of
sensor readings
per day.
Images, audio, locations, movements,,
heartbeats, stress, EEG, data interactions,
communications, learnings, activities…

28. But.. all this data is meaningless without
software to organise it…
An organisation you trust to maintain it.
And it to be always accessible.

29. There are a lot of
technical challenges to
be solved before a
functioning lifelogging
application is
operational.
This is an enormous
research challenge.
Segmentation
Find the unit of retrieval for many use-
cases… there is no one correct unit
Enrichment
Automatically turn raw
sensor data into
meaningful information
Search Engine
To index the data
Access Support
Supporting long-term use
with many use cases
Assuming you have Rich Sensing

30. Many events into one logical
semantic unit
Experience
4
5
A logical concatenation of moments
into a semantic unit (about 30 per
day)
Event
3
Multi-modal raw data.
Items
1
A short snapshot of activity
Moments
2
A summary, a reflection or an
analytical aggregation of data.
Abstraction
Segment into Units of Retrieval
Depends on the Information Need - which changes over time

31. An Event Segmentation Model

32. Another Event Segmentation Model

33. Raw$
Sensors$
What$doing$
What$
Environment$
Movement$
• Ac8vity$
• Energy$
Where$Who$is$there$
When$$
Why$
“Shopping for a coat last
Tuesday in Dublin”
Enrichment to add semantic value. Necessary for most
use-cases.

34. Develop Appropriate Search Engines
Ubiquitous Search
Contextual Search
Push & Pull Interaction
Browsing and Summarisation

35. And Understand the Use-Cases

36. PROGRESS
WHAT -VISION - HOW - PROGRESS - FUTURE

37. 1928
Bucky-
Fuller
1980s
Steve
Mann
2004
Williams
(Sensecam)
2015
First Gen
Tech.
2006+
Memory
Studies
2004
G. Bell
(MyLifeBits)
1946
Vannevar
Bush

38. 1928
Bucky-
Fuller
Dymaxion Chronofile
A very accurate physical record of a human
(every 15 minutes from 1920-1983)

39. 1946
Vannevar
Bush
MEMEX
A lifelog and hypermedia proposal

40. 1980s
Steve
Mann
Built a version of Google Glass 25 years ago.

41. 2004
G. Bell
(MyLifeBits)
Gordon Bell’s MyLifeBits, a first true lifelog.

42. 2004
Williams
(Sensecam)
SenseCam - up to 5,000 images per day

43. 2006+
Memory
Studies Berry, Hodges, Moulin, Doherty
First studies into the memory-support potential

44. 2015
First Gen
Tech.
Narrative Clip camera… 2,000 photos per day.

45. And a lot of early applications, by ourselves and others
Early insights into the nature of lifelog search engines

46. An Event Segmentation Model

47. Aizawa’s Foodlog

48. Reporter App
Reporter &The Annual Reports
Nick Feltron

49. SIMPLE ANALYTICS
QUANTIFIED SELF

50. DEEP ANALYTICS
UNDERSTAND WHATTHE USER EXPERIENCES
Find anything you have seen before. Objects, people, brands,
environments, etc…

51. EYEAWARE PLATFORM

52. COLOUR OF LIFE

53. !
!
SOCIETAL ANALYTICS

54. FUTURE
WHAT -VISION - HOW - PROGRESS - FUTURE

55. Lifelogging will create a whole new set of
opportunities & challenges for
industry and society… but
This is private data and it is not curated
Most of it will never be seen and the lifelogger will forget what
is there
Data security is vital with huge consequences
Data outlives the person
The digital self must be self-organising
Who can access the data (now and post-life)?

56. Many Opportunities
New Sensors
New Search Engines
New Scientific Challenges
Opportunities for Assistive Technologies
But Little Understanding of the Use-Cases

57. Privacy is a big issue…
Throughout life and afterwards.

58. The Privacy Paradox
Wearer or Bystander… who looses more privacy?

59. Some Thoughts for Digital Preservation
I don’t know what my digital self actually stores.
The data is incomplete and inaccurate
I have no time to curate it and I can’t easily manage it. It needs
to be self-curating and self-organising
The data will outlive the individual - Someday somebody will
look at it
Data becomes the truth

60. THANKYOU
@cathal & cathal@gmail.com
http://about.me/cgurrin/
Any Questions?
Thanks to the DCU team and our collaborators!
DCU 2015
L i f e l o g g i n g -
P e r s o n a l B i g
Data… from the
F o u n d a t i o n a n d
Trends in Information
Retrieval series. Free
d o w n l o a d , a s k
Google.
NTCIR-12
Lifelog task
First
comparative
evaluation
campaign
for lifelog
data and
search