Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Biohackers Summit 2015 - Lifelogging, a new era of Personal Data


Published on

My 40 minute presentation on lifelogging for the Biohackers Summit 2015

Published in: Technology
  • Be the first to comment

Biohackers Summit 2015 - Lifelogging, a new era of Personal Data

  1. 1. LIFELOGGING A NEW ERA OF PERSONAL DATA Dr. Cathal Gurrin (@cathal) lifelogger - researcher - educator Dublin City University & Insight Centre for Data Analytics Biohackers Summit 2015 24th September 2015
  2. 2. From Cave Paintings…
  3. 3. … to Diaries
  4. 4. Technology allows us to record our lives
 in previously unimaginable detail… … but why ?
  5. 5. Using mobile/wearable devices and information loggers to automatically record everything you see, hear, learn and experience. Creates a complete and accurate record of an individual - a Lifelog. Challenge is to extract value from this new data. Lifelogging
  6. 6. In the era of lifelogging, you will be able to summon up any memory or life experience… It will change the way we work and learn, improve our health, change relationships… It will change what it means to be human, and it is happening now. In fact, it is inevitable…
  7. 7. MyLifeBits & Sensecam Gordon Bell (Microsoft) 2004
  8. 8. Sense and analyse factors of interest through numbers to gain knowledge Using knowledge for self-improvement through experimentation Digitise as much as you can of life experience… for many reasons, mostly unknown… LifeloggingQuantified
 Self Biohacking Positioning my Research
  9. 9. Lifelogging can
 generate thousands of
 images per day, hours of audio/video
 and tens of thousands
 of sensor readings,
 interactions… The challenge is to automatically analyse this data and make it useful for the individual.
  10. 10. Quantified Self Enhanced Knowledge Power to Change Performance Enhancement Data for Empowerment New Insights Population-wide studies Healthcare Enhancement Enhancing Human Memory Upgraded Recall Assistive Technologies Enhanced Memory New Interactions Rich Sharing Data Partners/Carers Social Enhancements Why?To provide knowledge to empower… Quantified-Self Memory
  11. 11. Quantified Self Analytics BASISTracker Watch
  12. 12. Memory Enhancement RECALL/RETRIEVAL REFLECTION REMINISCENCE Quantified-Self Analytics with Limitless REFLECTION A Search Engine for Life Experience. Never Forget. RECALL/RETRIEVAL Reliving Past Memories for Personal Uses or Sharing. REMINISCENCE
  13. 13. The aim is to develop prototype memory upgrading software. An assistive technology that experiences what you experience and is always on and does not need any user input, except queries.
  14. 14. Sensory( Memory( Short-term( Memory( Long-term( Memory( Musical?( Explicit( (conscious)( Declara<ve( (events(and(facts)( Episodic( (events(and( experiences)( Seman<c( (concepts(&(facts)( Implicit( (unconscious)( Procedural(That means looking at human memory, and how it works… Episodic Memory, Query Mechanisms, etc…
  15. 15. And develop targeted applications… such as to build various memory models
  16. 16. ! ! Or large-scale tools to understand individuals and their activities… even on a societal level
  17. 17. Automatically annotate, enrich, link and store for future search, retrieval and access. Index Pervasive access to support Reminiscing, Reflection and Retrieval of Experiences Interact Automatically sense using a small set of wearable and informational sensors. Sense Automatically generate meaningful units of retrieval by modeling human memory. Segment Four Core Components are Required to build a Lifelogging Platform There are a lot of research challenges here, at every step. But they are all needed to develop a lifelogging platform technology.
  19. 19. There is no sensor that can record everything experience, in multiple dimensions
  20. 20. Autographer Panasonic 4K Google Glass Moves App Google Fit BASIS Watch Strava RescueTime LoggerMan Camlapse MyTracks OpenPaths CameraPhoneInstagram Media Lifelogging Activity Information Access SMS Backup CallRecorderPro VoiceRecorder WebServices Swarm Location 23&Me EEG Diet Others Health NarrativeClip Secure Personal Lifelog
  21. 21. The big challenge isVisual Lifelogging
  22. 22. It is not conventional photos, just data, 2,000 - 5,000 per day!Too many for an individual to analyse
  23. 23. My Lifelog in Numbers 70+ Papers and 12 first generation prototypes 10 Years of location log, with millions of GPS points 80 Million: heartbeats, with GSR and activity 1 Year 
 of computer interactions (mouse, keyboard) 9 Years of lifelog, since 2006 16.5 Million wearable camera images About 1TB per year
  25. 25. Segmentation of raw data into units such as events or moments. These can be enriched automatically with metadata, increasing their value. Events are analogous to our episodic memory
  26. 26. Event Detection Sharing Search API coming in the
 next few months Narrative Clip
  27. 27. Event Segmentation - Concept Filtering EyeAware Platform
  28. 28. Automatic Event Detection with Linkage & Browsing
  29. 29. Like all multimedia data, we began by browsing, but there is too much data, much repetition. We need search (Googlisation). 2.5 year study into locating important items: Increase from 25% to 75% success in 1/10th the time when searching not browsing. Searching is based on data analytics and machine/deep learning to ‘understand’ the sensor data.Segmentation Find the unit of retrieval for many use- cases… there is no one correct unit Enrichment Automatically turn raw sensor data into meaningful information Search Engine To index the data Interfaces Supporting Applications
  30. 30. Aiden Doherty, DCU, office setting, conversation, indoor,
 discussing CHI paper. “On Sept 23rd, I was in DCU discussing the CHI paper with Aiden at his desk” The challenge is to 
 automatically extract knowledge
 from the lifelog data to support
 recall/retrieval, reminiscence
 and reflection.
  31. 31. Raw$ Sensors$ What$doing$ What$ Environment$ Movement$ • Ac8vity$ • Energy$ Where$Who$is$there$ When$$ Why$ “Shopping for a coat last
 Tuesday in Helsinki” Enrich Semantics by Applying Data Analytics
  32. 32. Integrating AI - Deep Learning To understand what the user is seeing and doing
  33. 33. Kahneman et al.A survey method for characterizing daily life experience:The day reconstruction method. Science, 306(5702):1776–1780, 2004. 1 2 3 4 Intimate Relations 5 6 Socialising Relaxing Pray/Worship/Meditate Eating Exercising 7 WatchingTV 8 Shopping 9 10 11 12 Preparing Food 13 14 On the Phone Napping Taking Care of Children Computer/Internet Housework 15 Working 16 Commuting Recognising Life Activities
  34. 34. Summarisation REFLECTION Search Engine RECALL/RETRIEVAL Browsing REMINISCENCE Considering the Use-Cases and Developing Applications Amended from:Abigail J. Sellen and Steve Whittaker. 2010. Beyond total capture: a constructive critique of lifelogging. Commun.ACM 53, 5 (May 2010)
  35. 35. Recall / Retrieval - Prototype Search Engines
  36. 36. Reflecting on Life at a Glance - Colour of Life
  37. 37. Reflection for Enhancing Self Awareness
  38. 38. Reminiscence Supporting Digital Memory
  39. 39. How does this relate to BioHacking? There is a lot of R&D still to do… no consideration of UI No adequate lifelogging device or software yet Privacy (lifelogging looks out, QS looks in) Trust for sharing and storage Get data now, you can not get data retrospectively New tools and software will extract value later Some Final Thoughts
  40. 40. Privacy Awareness - Automated Negative Face Blurring with real-time Policy-driven Access Restrictions Privacy
  41. 41. THANK YOU Cathal Gurrin @cathal Any Questions? Interested in working with us, let me know… “ L i f e l o g g i n g - Personal Big Data” from the Foundation a n d Tr e n d s i n Information Retrieval s e r i e s . F r e e d o w n l o a d , a s k Google.