Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Emotive Media - Visualization and Analysis of Human Bio-Feedback Data

265 views

Published on

Emotive Media are media environments where an emotional channel is added in addition to other transmission channels in computer mediated communication. The medium actively recognizes human emotions, and is capable of simulating emotions as well. In this presentation, very basic aspects of emotive media are presented. The talk layouts the basics through an investigation of methods emerging from affective computation, brain-interfaces, and cognitive concepts. Practical examples, such as cognitive Big Data, the LudoViCo UX Machine, Empathy Interfaces, Portable Personality (P2), Financial Texts Mining for Sentiment Analysis, and Visualization are presented. More information (in particular publications) can be found on www.artur-lugmayr.com.

Published in: Science
  • Be the first to comment

Emotive Media - Visualization and Analysis of Human Bio-Feedback Data

  1. 1. Emotive Media Visualisation and Analysis of Human Bio-Feedback Data 26/08/2016 @ Curtin HIVE, Perth, Australia A/Prof. Artur Lugmayr www.artur-lugmayr.com artur.lugmayr@artur-lugmayr.com
  2. 2. https://www.fitbit.com/au, http://emotiv.com/
  3. 3. Trigger Peak of Expectations Disillusion Enlightenment Plateau of Productivity Speech Recognition Artificial Intelligence Deep Learning Open AI Gesture Recognition Automated Narratives Traditional Media IoT Affective Computation Smart Home Smart Accessories Brain Interfaces Sensor Interfaces Biosensors Big Data NOW 2-5 YEARS 5-10 YEARS > 10 YEARS Technologies – Between Hype and Maturity QoE (Quality of Experience) Intelligent Machines
  4. 4. Practical Applications • Health care and wellness • eLearning • Robotics and digital pets • Identification and safety • Sentiment and text analysis • Small gadgets and wearables • Sports analysis • User-experience research • Quality of Experience (QoE) • Analysis of archives and media • Advertisement analytics • Game behaviour and analytics • Organizational management • Intelligent machines • …
  5. 5. Affective Computation Neurosciences Media Studies and Communication Theories User Experience, and Human- Computer- Interaction Psychology of ‘The Human Mind’ Computational Models Involved Research Fields
  6. 6. http://muvis.cs.tut.fi/index.html Previously…. the conveyer belt for data processing
  7. 7. In the Future… we need a ‘cognitive’ conveyer belt for knowledge and wisdom
  8. 8. · Situation · Object · Collaboration · Behaviour · Perception · Attention · Arousal · Emotion · Experience · Feelings · Cognition Mind Communication Cause · Language · Personality · Aesthetics · Emotional Binding · Artificial Intelligence · Motives and Believes Concepts The Key in Emotional Media are Higher Level Concepts, as e.g. Language, Aesthetics, and Personality Lugmayr, Artur, Tillmann Dorsch and Pabo Roman Humanes. "Emotional Ambient Media." Handbook of Research on Synthetic Emotions and Sociable Robotics: New Applications in Affective Computing and Artificial Intelligence. IGI Global, 2009. 443-459. Web. 27 Aug. 2016. doi:10.4018/978-1-60566-354- 8.ch022
  9. 9. Emotional State Signal Level (Signals) Concept Level (Concepts) Symbol Level (Symbols) Pattern Level (Patterns) ActionMind Reaction Level CAPTURE SIMULATE CONCEPT RECOGNIZE LEARN COMMUNICATE Emotional Media are Dual – Media that Understand Emotions, and Media that Simulate Emotions “getting computers to ‘think’ like humans, [and] AI’s next natural step seems to be getting computers to ‘feel’ like humans” J. Krikke and B. Alfonsi, "In the News," IntelligentSystems, IEEE, vol. 21, pp. 102-104, c3, 2006
  10. 10. • Hayley Hung, Hatice Gunes, Introduction to Social Signal Processing, ACM Multimedia 2015: Emotional and Social Signals Tutorial • New Social Signals in a New Interaction World: The Next Frontier for Social Signal Processing, A Vinciarelli, AS Pentland, Systems, Man, and Cybernetics Magazine, IEEE 1(2), 2015 “A Social Signal is a communicative or informative signal that, either directly or indirectly provides information concerning social interactions, social emotions, social attitudes or social relations” • http://www.theaustralian.com.au/news/d eputy-opposition-leader-julie-bishop-says- peter-slipper-cannot-remain-as- speaker/story-e6frg6n6-1226491792937 • http://www.abc.net.au/pm/content/2015 /s4331627.htm NOTE! Due to eventual copyright reasons, I removed 2 images of two Australian Politicians. You want to know what was here? Click on the links below…
  11. 11. Semantics & Meaning (Stock Exchange Sentiment) Macroeconomic Indicators Interest rate Inflation Domestic credit Industrial production Unemployment rate Consumer sentiment Consumer prices Growth durable goods GDP …. Implicit Sentiment Indicators Call/Put ratio Open interest IPOs IPO first day returns Mutual fund flows … Explicit Sentiment Indicators Boerse Frankfurt sentiment index Economic sentiment index (ESI) ZEW Indicator Ifo Business Climate Index … Textual Sentiment Sources Sources Newsletters RSS Feeds News … Sentiment Indicator Sentiment index Trend and trend strength (Resistance/Support) (Hot news) Textual Sentiment Analysis Text Mining Sentiment Aggregation Sentiment Model Technical Indicators Volatility Volume Turnover RSI, CCI, ARMS, … …. Lugmayr and G. Gossen, “Evaluation of methods and techniques for language based sentiment analysis for DAX30 stock exchange - a first concept of a "LUGO" sentiment indicator,” in Proceedings of the 5th International Workshop on Semantic Ambient Media Experience (SAME) - in Conjunction with Pervasive 2012, A. Lugmayr, T. Risse, B. Stockleben, J. Kaario, B. Pogorelc, and E. S. Asensio, Eds. Newcastle, UK: Tampere University of Technology (TUT),
  12. 12. Techniques in Social Signal Recognition • Verbal communication • Non-verbal communication • Gestures, facial expressions, postures, gaze, pupils, vocal behaviour, speech activity, personal space, seating arrangements, … • Media archive analytics • Multimedia content analysis, low-level features, semantic relations, … • Social media analytics • Media achieve analytics • Hayley Hung, Hatice Gunes, Introduction to Social Signal Processing, ACM Multimedia 2015: Emotional and Social Signals Tutorial
  13. 13. Bio-Feedback Visualization https://processing.org/ Examples: https://processing.org/examples/ https://www.youtube.com/watch?v=SWCkEKrgaXk NOTE! Due to eventual copyright reasons, I removed 2 images of the example. You want to know what was here? Click on the links below…
  14. 14. Emotional Interaction and Interfaces Lugmayr, A., Dorsch, T. & Humanes, P.R., 2009. Emotional Ambient Media. In Handbook of Research on Synthetic Emotions and Sociable Robotics: New Applications in Affective Computing and Artificial Intelligence. IGI Global.
  15. 15. Empathy Glasses (CHI 2016) • Masai, Katsutoshi et al. “Empathy Glasses.” CHI (2016)., https://www.semanticscholar.org/paper/Empathy-Glasses-Masai- Kunze/8cc9f1a26ef16490d4688bb9928fe91e3f71d02f NOTE! Due to eventual copyright reasons, I removed the images of the example. You want to know what was here? Click on the links below…
  16. 16. Personality Personality Personality Personality Personality Personality Personality Personality Personality Collaborative Space Personality Computation Bits of personality everywhere (Portable Personality – P2), similar to Microsoft’s Cortana and Apple’s Siri NOTE! Due to eventual copyright reasons, I removed 2 images of the example. You want to know what was here? Click on the links below… P2 is similar to the film idea in ‘The Golden Compass’. Check it out: https://en.wikipedia.or g/wiki/The_Golden_Co mpass_(film) S. Reymann, V. Bruns, and A. Lugmayr, "P2 - Portable Personality a Middleware Solution for Smart User Profile Management and Distribution," in Interactive TV: A Shared Experience, TISCP Adjunct Proceedings of EuroITV 2007, Amsterdam, 2007 S. Uhlmann and A. Lugmayr. Media in the Ubiquitous Era: Ambient, Social and Gaming Media, chapter Portable Personality and its Personalization Algorithms: An Overview and Directions, pages 66–93. IGI Global, Hershey, USA, 2011. ISBN 9781609607746.
  17. 17. P2 – Portable Personality www.portablepersonality.org Provider P2 Service P2 ServiceP2 Mobile Service Consumer Service S. Reymann, V. Bruns, and A. Lugmayr, "P2 - Portable Personality a Middleware Solution for Smart User Profile Management and Distribution," in Interactive TV: A Shared Experience, TISCP Adjunct Proceedings of EuroITV 2007, Amsterdam, 2007 S. Uhlmann and A. Lugmayr. Media in the Ubiquitous Era: Ambient, Social and Gaming Media, chapter Portable Personality and its Personalization Algorithms: An Overview and Directions, pages 66–93. IGI Global, Hershey, USA, 2011. ISBN 9781609607746.
  18. 18. Free UX Testing Tool: The LudoVico UX Machine for Physiological Sensor Data Recording, Analysis, and Visualization for User Experience Design Experiments, SEACHI Workshop, SIGCHI, 2016 The Ludovico UX Machine WWW.UX-MACHINE.COM (freely available sometimes in 2016)
  19. 19. The LudoVico UX Machine Medical Sensor Data: Collection and interpretation of medical sensor data, such as EMC, EMG, breath, pulse, body temperature, body position, and galvinistic skin response. Smart Environment Sensor Data: data coming from smart city environments, mashup data, location based data, mobile phone data. Body Motion Sensor Data: Collection and interpretation of body motion sensor data through multiple sensor attached to a human body. Extendible Platform: A plug-in concept and a light-weight sensor data protocol allows high extendibility of the platform, add new sensor types, additional features, new analysis techniques, and new functionality. Freely Available: The LudoViko UX Tool is freely available for download: http://www.artur-lugmayr.com starting from mid-2016. Subscribe to our newsletter to get more information.
  20. 20. The Key is Visualisation Data Analysis Re- presentation Presentation Interaction Cognition & Perception Communication Context
  21. 21. VALENCE AROUSAL LOW (passive) HIGH (active) NEGATIVE POSITIVE … and developing user-interfaces that people understand in particular domains…. …a film-maker might not understand complex time-series data in form of diagrams, but a user- interface that he is comfortable with as e.g. Adobe Premiere integrating an ‘emotional monitor’…
  22. 22. Marshall McLuhan • Electronic culture as follow up culture for 400 years of print culture • Media extend human senses: sight, hearing, touch, and smell • Media technologies change society and culture: extensions of our bodies change mind & society affecting our mental functions • generating ideas, perception of the world, emotions, experiences • The effect of McLuhan’s thesis: • Technology DETERMINES social outcome • Technology enables man extended and a live model outside the mind • Cybernetics • Examples: light bulb, hammer, 3D cinema, ambient media ... https://en.wikipedia.org/wiki/Marshall_McLuhan 1964 Understanding Media: The Extensions of Man; 1st ed. McGraw Hill, NY; reissued by MIT Press, 1994, with introduction by Lewis H. Lapham; reissued by Gingko Press, 2003. ISBN 1- 58423-073-8.

×