This document discusses human sensing, tagging, and interaction. It describes how sensing human behavior through devices can help understand human activities, support activities in real-time, and automatically tag activities for future retrieval. Technologies discussed include sensing text, audio, video, activities, environments, and physiological data. Tagging can be done manually, semi-automatically, or fully automatically. The document also discusses interpreting interactions and using the data for question answering and dialogue systems. Overall it explores how collecting human behavior data through sensing can improve understanding, support, and retrieval of human information and interactions.
Building the Knowledge of Human Perception into E-LearningShalin Hai-Jew
Human perceptual systems—for sight, sound, taste, touch, and smell (and maybe even embodied proprioception)—may offer some guidelines for how to build multimedia e-learning: immersive 3D simulations, imagery for analysis, sight-and-sound distributions of information channels, and other applications. This will offer a brief overview of human perception (with a little human cognition thrown in) and some light applications to the design of e-learning.
The document discusses theories of objects from STS scholars like Heidegger, Ihde, and Latour, and how they can inform our understanding of objects in an Internet of Things world. It explores how STS theories view the composition, position, and relations of objects. The researchers are working on a model and application to implement objects in IoT, and are drawing on lessons from these theorists about how technologies mediate our relations with objects and environments. They are still working through questions about how users can define, share, and search for IoT objects.
The document discusses Blue Eyes technology, which aims to give computers human-like perceptual abilities such as sight, hearing, and touch. This would allow computers to better interact with and understand humans. The technology uses sensors to identify a user's actions, physical and emotional states. It analyzes this information to help the user or perform expected tasks. For example, a TV could turn on when detecting eye contact. The goal is to create devices with emotional intelligence that can recognize and respond to human emotions during interactions.
The document discusses how human intelligence evolved to understand the physical world through objects rather than the digital world. It argues that physicality is the most basic and universal language for humans, as our brains developed over millions of years to intuitively understand the complex meanings and emotions conveyed by textures, materials, and other physical properties of objects. In the digital age, there is a clash between our innate understanding of the physical and the virtual interactions promoted by new technologies. The document advocates for communicating through physical objects and artifacts that tap into humans' hardwired ability to interpret the physical world.
Objects tell human stories — real things connect people to ideasARTOMATIC
Objects tell human stories is a document that argues objects are better than words and pictures alone for communicating ideas and stories in the digital age. It discusses how objects have historically been used to document human achievements and provide intuitive, emotional context that informs how people understand information. The document advocates using real, physical objects to tell stories as they provide a universal language that engages people on an emotional, tactile level better than digital information alone.
This document summarizes context-aware artifacts and mobile phones. It discusses two approaches for context-aware artifacts: self-supported and infrastructure-supported. It provides examples of self-supported artifacts like smart furniture, mirrors, and chairs that have embedded sensors. It also discusses how infrastructure can perceive contexts related to artifacts. Examples of context-aware mobile phones are described that use self-contained sensors or external infrastructure for context-awareness. Frameworks like ContextPhone and Situation-Aware Phone that enable context-aware applications on mobile devices are summarized.
The document discusses the topics of artificial intelligence and expert systems. It defines AI as creating machine intelligence that can perceive its environment and take actions to maximize success. Expert systems are described as computer systems that emulate human expert decision making through storing and reasoning about knowledge in a domain. The document also notes that while progress has been made in games playing, natural language processing remains a challenging area of AI that could allow more natural human-computer interaction if solved.
Kognit – Cognitive Assistants for Dementia Patientsdiannepatricia
Daniel Sonntag from the German Research Center for Artificial Intelligence is making this presentation at the Cognitive Systems Institute Group Speaker Series on October 29, 2015.
Building the Knowledge of Human Perception into E-LearningShalin Hai-Jew
Human perceptual systems—for sight, sound, taste, touch, and smell (and maybe even embodied proprioception)—may offer some guidelines for how to build multimedia e-learning: immersive 3D simulations, imagery for analysis, sight-and-sound distributions of information channels, and other applications. This will offer a brief overview of human perception (with a little human cognition thrown in) and some light applications to the design of e-learning.
The document discusses theories of objects from STS scholars like Heidegger, Ihde, and Latour, and how they can inform our understanding of objects in an Internet of Things world. It explores how STS theories view the composition, position, and relations of objects. The researchers are working on a model and application to implement objects in IoT, and are drawing on lessons from these theorists about how technologies mediate our relations with objects and environments. They are still working through questions about how users can define, share, and search for IoT objects.
The document discusses Blue Eyes technology, which aims to give computers human-like perceptual abilities such as sight, hearing, and touch. This would allow computers to better interact with and understand humans. The technology uses sensors to identify a user's actions, physical and emotional states. It analyzes this information to help the user or perform expected tasks. For example, a TV could turn on when detecting eye contact. The goal is to create devices with emotional intelligence that can recognize and respond to human emotions during interactions.
The document discusses how human intelligence evolved to understand the physical world through objects rather than the digital world. It argues that physicality is the most basic and universal language for humans, as our brains developed over millions of years to intuitively understand the complex meanings and emotions conveyed by textures, materials, and other physical properties of objects. In the digital age, there is a clash between our innate understanding of the physical and the virtual interactions promoted by new technologies. The document advocates for communicating through physical objects and artifacts that tap into humans' hardwired ability to interpret the physical world.
Objects tell human stories — real things connect people to ideasARTOMATIC
Objects tell human stories is a document that argues objects are better than words and pictures alone for communicating ideas and stories in the digital age. It discusses how objects have historically been used to document human achievements and provide intuitive, emotional context that informs how people understand information. The document advocates using real, physical objects to tell stories as they provide a universal language that engages people on an emotional, tactile level better than digital information alone.
This document summarizes context-aware artifacts and mobile phones. It discusses two approaches for context-aware artifacts: self-supported and infrastructure-supported. It provides examples of self-supported artifacts like smart furniture, mirrors, and chairs that have embedded sensors. It also discusses how infrastructure can perceive contexts related to artifacts. Examples of context-aware mobile phones are described that use self-contained sensors or external infrastructure for context-awareness. Frameworks like ContextPhone and Situation-Aware Phone that enable context-aware applications on mobile devices are summarized.
The document discusses the topics of artificial intelligence and expert systems. It defines AI as creating machine intelligence that can perceive its environment and take actions to maximize success. Expert systems are described as computer systems that emulate human expert decision making through storing and reasoning about knowledge in a domain. The document also notes that while progress has been made in games playing, natural language processing remains a challenging area of AI that could allow more natural human-computer interaction if solved.
Kognit – Cognitive Assistants for Dementia Patientsdiannepatricia
Daniel Sonntag from the German Research Center for Artificial Intelligence is making this presentation at the Cognitive Systems Institute Group Speaker Series on October 29, 2015.
This document describes the components of a generic virtual reality system. It discusses the four main systems: the virtual environment, computer environment, VR technology, and modes of interaction. It provides details on the virtual environment components like virtual objects, lights, animation, physical simulation, level of detail and collision detection. It also describes the computer environment components like the processor, I/O channels, VE database and real-time operating system. Finally, it discusses the hardware used for VR technology like head tracking, image display, sound and haptics as well as interaction modes like gestures and interfaces.
1. The document discusses new technologies that may impact future game development, including motion sensors, multi-touch screens, gyroscopes, phantom sensation, electromyography, and artificial intelligence.
2. It outlines the group's project workflow, including brainstorming ideas, conducting research, creating a website and presentation.
3. The document introduces several emerging technologies, such as motion sensing and multi-touch, that could revolutionize game input methods and user interaction.
The document describes the Sixth Sense device, a wearable gestural interface developed by Pranav Mistry. It consists of a camera, projector, and mirror coupled in a pendant-like mobile device. The camera tracks hand gestures which are processed by a smartphone to project information onto surfaces. Some applications include making calls, getting maps/product info, and interacting with projected interfaces through natural hand motions. The Sixth Sense aims to augment physical reality with digital information accessible through gestures.
This document discusses adaptive behavior and higher cognitive functions from a multidisciplinary perspective, focusing on the social factors that make humans unique. It compares humans, chimpanzees, and rhesus macaques in terms of genetics, brain structure, and social intelligence. While humans and chimpanzees share more genetic similarities, rhesus macaques have social behaviors more like humans. The field of social neuroscience examines how the brain mediates social interactions and behaviors through structures involved in mentalizing and empathizing. Understanding primate social organizations provides insights into the evolution of human societies driven by social intelligence.
Textsl: a screen reader accessible virtual world client for second lifeEelke Folmer
Virtual worlds are not accessible to users who are visually impaired as they lack any textual representation that can be read with a screen reader. We developed an interface modeled after text based adventure games like zork that allows a screen reader user to iteratively interact with the popular virtual world of second life.
Smartphone and tablet apps for people with disabilities jemsshep07
This presentation explains assistive technology, accessibility and universal design with regard to smartphones and tablets. It also presents a list of recommended apps for case managers and people with disabilities.
Dice01 re life-ict-system-smartdiagn-pdw-27june2013Jun Hu
1) The document discusses an ICT system called ReLifE that aims to create a smart and remote diagnosis infrastructure in the Netherlands.
2) ReLifE seeks to shift healthcare from hospitals to primary care and self-care through unobtrusive diagnostic methods enabled by technologies like sensing, imaging, and ICT.
3) It proposes a conceptual person-centric infrastructure that empowers individuals and connects all stakeholders through a shared data platform and applications while maintaining privacy, security, and patient control of personal data.
This document summarizes key insights about user experience design for interactive systems and everyday life. It discusses how the true magic of magical objects in Harry Potter stems from the experiences they enable rather than their underlying technology. It also outlines three main insights for experience design: 1) seeing value in supporting user experiences rather than technology, 2) understanding how design can help satisfy basic human needs, and 3) seamless integration of technology into everyday life so it informs users without overburdening them. The document advocates for calm, peripheral technologies that fade into the background of activities.
The document discusses the benefits of exercise for mental health. Regular physical activity can help reduce anxiety and depression and improve mood and cognitive function. Exercise causes chemical changes in the brain that may help protect against mental illness and improve symptoms for those who already suffer from conditions like anxiety and depression.
Here are the key points about discounting:
- Discounting refers to exponentially decreasing the value of rewards received in the future, compared to rewards in the present. This is done with a discount factor γ between 0 and 1.
- Discounting encourages agents to prefer sooner rewards over later rewards. It helps algorithms converge by ensuring the value of future rewards is finite.
- With discounting, the utility of a sequence of rewards is calculated as:
U = R1 + γR2 + γ^2R3 + ...
- Discounting leads to stationary preferences over reward sequences, meaning the preferences don't change over time. The two ways to define utilities under stationary preferences are additive utility and discounted utility.
From technology development_towards_social_innovation [compatibility mode]Jun Hu
Dr. Boris De Ruyter discusses the transition from technology development to social innovation in ambient intelligence research. Over time, the focus has shifted from developing new technologies to studying how technology impacts user experiences and society. Dr. De Ruyter's research at Philips examines these experiences and effects through context studies, laboratory studies, and field studies to gain user insights and validate the longer term social and experiential impacts of emerging technologies. Examples of this research include studying the effects of mimicry and social praise on interactions with artificial agents, and exploring how lighting impacts sleep in hospital settings.
The document discusses the rise of "data-ism" and our increasing ability to gather huge amounts of data, though only a small percentage is analyzed. It also addresses the cultural assumptions that come with this, such as the idea that everything should be measured and that data provides an objective lens. The document advocates for a more balanced approach and being mindful of both the benefits and potential limitations of technology. It also presents examples of individuals like Armand Schulthess who were early "data gardeners" tending to information before the digital age.
This document provides an overview of visual sensor networks and smart cameras. It discusses sensor networks and how they use wireless communication between small battery-powered devices to collect and transmit sensor data over multiple hops. It then describes smart cameras, which combine sensing, processing, and communication to perform image analysis and collaboration. Key characteristics of visual sensor networks are discussed, including resource limitations, on-board processing, real-time operation, and autonomous camera collaboration. Several problems in visual sensor networks are also outlined, such as sensor placement, synchronization, and data distribution. Finally, examples of centralized and distributed configuration methods and a market-based approach to tracking handovers between cameras are presented.
1. The document discusses using lighting design and technology to enhance natural lighting experiences and provide health benefits. It explores how light affects behaviors, moods, and circadian rhythms.
2. Various solutions are proposed, including lighting designs that mimic natural light patterns, interactive lighting systems, and lighting systems for outdoor spaces that consider different user needs over the course of a day.
3. Case studies demonstrate solutions such as lighting designs for bicycle paths tailored to different times of day, lighting installations for festivals, and ambient intelligent hospital rooms that adapt lighting conditions to support patient recovery.
Cenicientos antes solían ir a las tiendas para comprar pero ahora compran desde casa por Internet. Los niños jugaban en la calle antes pero ahora juegan más a juegos por Internet.
1. The document discusses improving work spaces to focus on work, be comfortable, inspire creativity, have fast network coverage, improve space utilization, and facilitate easy communication and sharing.
2. Examples of work spaces from Alibaba, Douban, and Innovation Works are provided that include open work areas, meeting rooms, coffee bars, and rest areas.
3. Applications that can help working like time planning, task management, meeting records, mind mapping, messaging, and collaboration platforms are suggested. The most important thing is to improve efficiency.
O Girassol - 2016 1º trimestre - jornal do lar - 6ª EdiçãoJoel Pacheco
O documento resume as atividades realizadas no Lar durante o segundo trimestre de 2016, incluindo ginástica semanal, parabéns para duas residentes, apresentações do grupo coral, passeios, visitas de outros lares e trabalhos manuais realizados pelos residentes.
The document discusses the relationship between design practice and design research. It notes there is often a gap between theory and practice in design. Some key points discussed include:
- Traditional PhD models may not fully support design practice-based research
- Design research could benefit from a more integrated interplay between practice and reflection like in other fields
- Design projects may contribute new knowledge and insights similar to experiments in other disciplines if properly structured and analyzed
It provides examples of practice-based design PhD theses and hypothesizes that design projects could provide insights like experiments if approached from multiple perspectives rather than just hypothesis testing.
FOVEA. Report. Monitor. Earn.
Fotografa un problema stradale, aggiungi una breve descrizione e posizionalo sulla mappa.
Seguilo nel tempo fino a che non è stato risolto e guadagna punti che ti permettono di ottenere ricompense nella vita reale.
This document describes the components of a generic virtual reality system. It discusses the four main systems: the virtual environment, computer environment, VR technology, and modes of interaction. It provides details on the virtual environment components like virtual objects, lights, animation, physical simulation, level of detail and collision detection. It also describes the computer environment components like the processor, I/O channels, VE database and real-time operating system. Finally, it discusses the hardware used for VR technology like head tracking, image display, sound and haptics as well as interaction modes like gestures and interfaces.
1. The document discusses new technologies that may impact future game development, including motion sensors, multi-touch screens, gyroscopes, phantom sensation, electromyography, and artificial intelligence.
2. It outlines the group's project workflow, including brainstorming ideas, conducting research, creating a website and presentation.
3. The document introduces several emerging technologies, such as motion sensing and multi-touch, that could revolutionize game input methods and user interaction.
The document describes the Sixth Sense device, a wearable gestural interface developed by Pranav Mistry. It consists of a camera, projector, and mirror coupled in a pendant-like mobile device. The camera tracks hand gestures which are processed by a smartphone to project information onto surfaces. Some applications include making calls, getting maps/product info, and interacting with projected interfaces through natural hand motions. The Sixth Sense aims to augment physical reality with digital information accessible through gestures.
This document discusses adaptive behavior and higher cognitive functions from a multidisciplinary perspective, focusing on the social factors that make humans unique. It compares humans, chimpanzees, and rhesus macaques in terms of genetics, brain structure, and social intelligence. While humans and chimpanzees share more genetic similarities, rhesus macaques have social behaviors more like humans. The field of social neuroscience examines how the brain mediates social interactions and behaviors through structures involved in mentalizing and empathizing. Understanding primate social organizations provides insights into the evolution of human societies driven by social intelligence.
Textsl: a screen reader accessible virtual world client for second lifeEelke Folmer
Virtual worlds are not accessible to users who are visually impaired as they lack any textual representation that can be read with a screen reader. We developed an interface modeled after text based adventure games like zork that allows a screen reader user to iteratively interact with the popular virtual world of second life.
Smartphone and tablet apps for people with disabilities jemsshep07
This presentation explains assistive technology, accessibility and universal design with regard to smartphones and tablets. It also presents a list of recommended apps for case managers and people with disabilities.
Dice01 re life-ict-system-smartdiagn-pdw-27june2013Jun Hu
1) The document discusses an ICT system called ReLifE that aims to create a smart and remote diagnosis infrastructure in the Netherlands.
2) ReLifE seeks to shift healthcare from hospitals to primary care and self-care through unobtrusive diagnostic methods enabled by technologies like sensing, imaging, and ICT.
3) It proposes a conceptual person-centric infrastructure that empowers individuals and connects all stakeholders through a shared data platform and applications while maintaining privacy, security, and patient control of personal data.
This document summarizes key insights about user experience design for interactive systems and everyday life. It discusses how the true magic of magical objects in Harry Potter stems from the experiences they enable rather than their underlying technology. It also outlines three main insights for experience design: 1) seeing value in supporting user experiences rather than technology, 2) understanding how design can help satisfy basic human needs, and 3) seamless integration of technology into everyday life so it informs users without overburdening them. The document advocates for calm, peripheral technologies that fade into the background of activities.
The document discusses the benefits of exercise for mental health. Regular physical activity can help reduce anxiety and depression and improve mood and cognitive function. Exercise causes chemical changes in the brain that may help protect against mental illness and improve symptoms for those who already suffer from conditions like anxiety and depression.
Here are the key points about discounting:
- Discounting refers to exponentially decreasing the value of rewards received in the future, compared to rewards in the present. This is done with a discount factor γ between 0 and 1.
- Discounting encourages agents to prefer sooner rewards over later rewards. It helps algorithms converge by ensuring the value of future rewards is finite.
- With discounting, the utility of a sequence of rewards is calculated as:
U = R1 + γR2 + γ^2R3 + ...
- Discounting leads to stationary preferences over reward sequences, meaning the preferences don't change over time. The two ways to define utilities under stationary preferences are additive utility and discounted utility.
From technology development_towards_social_innovation [compatibility mode]Jun Hu
Dr. Boris De Ruyter discusses the transition from technology development to social innovation in ambient intelligence research. Over time, the focus has shifted from developing new technologies to studying how technology impacts user experiences and society. Dr. De Ruyter's research at Philips examines these experiences and effects through context studies, laboratory studies, and field studies to gain user insights and validate the longer term social and experiential impacts of emerging technologies. Examples of this research include studying the effects of mimicry and social praise on interactions with artificial agents, and exploring how lighting impacts sleep in hospital settings.
The document discusses the rise of "data-ism" and our increasing ability to gather huge amounts of data, though only a small percentage is analyzed. It also addresses the cultural assumptions that come with this, such as the idea that everything should be measured and that data provides an objective lens. The document advocates for a more balanced approach and being mindful of both the benefits and potential limitations of technology. It also presents examples of individuals like Armand Schulthess who were early "data gardeners" tending to information before the digital age.
This document provides an overview of visual sensor networks and smart cameras. It discusses sensor networks and how they use wireless communication between small battery-powered devices to collect and transmit sensor data over multiple hops. It then describes smart cameras, which combine sensing, processing, and communication to perform image analysis and collaboration. Key characteristics of visual sensor networks are discussed, including resource limitations, on-board processing, real-time operation, and autonomous camera collaboration. Several problems in visual sensor networks are also outlined, such as sensor placement, synchronization, and data distribution. Finally, examples of centralized and distributed configuration methods and a market-based approach to tracking handovers between cameras are presented.
1. The document discusses using lighting design and technology to enhance natural lighting experiences and provide health benefits. It explores how light affects behaviors, moods, and circadian rhythms.
2. Various solutions are proposed, including lighting designs that mimic natural light patterns, interactive lighting systems, and lighting systems for outdoor spaces that consider different user needs over the course of a day.
3. Case studies demonstrate solutions such as lighting designs for bicycle paths tailored to different times of day, lighting installations for festivals, and ambient intelligent hospital rooms that adapt lighting conditions to support patient recovery.
Cenicientos antes solían ir a las tiendas para comprar pero ahora compran desde casa por Internet. Los niños jugaban en la calle antes pero ahora juegan más a juegos por Internet.
1. The document discusses improving work spaces to focus on work, be comfortable, inspire creativity, have fast network coverage, improve space utilization, and facilitate easy communication and sharing.
2. Examples of work spaces from Alibaba, Douban, and Innovation Works are provided that include open work areas, meeting rooms, coffee bars, and rest areas.
3. Applications that can help working like time planning, task management, meeting records, mind mapping, messaging, and collaboration platforms are suggested. The most important thing is to improve efficiency.
O Girassol - 2016 1º trimestre - jornal do lar - 6ª EdiçãoJoel Pacheco
O documento resume as atividades realizadas no Lar durante o segundo trimestre de 2016, incluindo ginástica semanal, parabéns para duas residentes, apresentações do grupo coral, passeios, visitas de outros lares e trabalhos manuais realizados pelos residentes.
The document discusses the relationship between design practice and design research. It notes there is often a gap between theory and practice in design. Some key points discussed include:
- Traditional PhD models may not fully support design practice-based research
- Design research could benefit from a more integrated interplay between practice and reflection like in other fields
- Design projects may contribute new knowledge and insights similar to experiments in other disciplines if properly structured and analyzed
It provides examples of practice-based design PhD theses and hypothesizes that design projects could provide insights like experiments if approached from multiple perspectives rather than just hypothesis testing.
FOVEA. Report. Monitor. Earn.
Fotografa un problema stradale, aggiungi una breve descrizione e posizionalo sulla mappa.
Seguilo nel tempo fino a che non è stato risolto e guadagna punti che ti permettono di ottenere ricompense nella vita reale.
A 1-hour introductory lecture on multimodal interaction that I gave to bachelor HCI students. Included a section on how to get started in this exciting line of research.
Science has made tremendous progress in explaining previously unexplained human phenomena through advances in data collection, processing, and machine learning. This document explores how human emotions, intuition, and first impressions could potentially be explained algorithmically and replicated by computers. While computers now surpass humans in speed and scale of certain logical decision-making, humans still have an edge in accuracy due to our strong contextual understanding from our five senses and upbringing. For machines to truly match human-level responses, they would need technology that can assimilate context as deeply as human sensory perception and rules informed by human ethics and morality.
This document discusses designing for the Internet of Things (IoT). It begins by defining the IoT as networks of physical objects with embedded sensors and actuators that communicate with other objects, databases, and people. It then discusses some challenges in designing for the IoT, including creating new interaction paradigms that leverage sensing capabilities while accommodating human behaviors. The document outlines characteristics of natural user interfaces for the IoT, such as considering context, cognitive load, social aspects, and movement. It provides examples of techniques for designing IoT interfaces, like bodystorming, gestural studies, prototyping, and usability testing.
This document is a coursework analysis report for an Interaction Design course. It discusses cognitive processes that affect how users interact with systems. It covers cognition, cognitive frameworks, and how to evaluate cognitive processes. It then discusses design principles, methodology, and multimedia used for a university website prototype for new students. The prototype is evaluated using use case diagrams, scenarios, prototypes, heuristic checks, and testing. The report provides an overview of the coursework project, which involved designing and evaluating a prototype for a university website using cognitive and interaction design concepts.
This document provides an overview of key concepts in human-computer interaction (HCI). It discusses three main areas: 1) definitions of HCI and why it is important, focusing on usability and accessibility, 2) aspects of human cognition relevant to design such as perception, memory, problem solving, and individual differences, and 3) direct applications of psychology principles to interface design. The goal is to understand users and incorporate findings from cognitive psychology into interactive systems.
1) The document discusses using wearable sensors and augmented reality to capture expert experiences and knowledge and allow trainees to re-enact and experience those experiences.
2) Key aspects that are captured include gaze direction, videos, audio, gestures, physiological data, and positioning of the expert in an environment. This data is stored and synchronized to create an experience recording.
3) Trainees can then re-enact the expert's experience in real-time by having the expert's contextualized data augmented onto their own experience through methods like displaying the expert's position or where they were looking. This allows the trainee to experience the presence of the expert during tasks.
This document discusses social virtual reality (SVR) and its potential applications. It begins by providing context on the history and development of VR. It then discusses how SVR allows for social interactions in virtual spaces through features like avatars, virtual environments, objects, and verbal/non-verbal communication. Specific examples are given of how avatar characteristics and virtual spaces can influence behaviors in VR and in real life through effects like the Proteus Effect. The document also explores potential uses of SVR in areas like education, collaboration, marketing, and a project implementing social extended reality in construction.
Crane and Shapiro present differing views on cognitive science and embodied cognition. Crane views cognitive science as focusing on computational processes in the brain. Shapiro argues embodied cognition either supplements or contradicts standard cognitive science. He lays out three hypotheses: conceptualization suggests embodiment adds to cognition; replacement claims embodiment replaces internal representations; constitution establishes cognition is constituted by embodied interactions in the world. The authors disagree on the role of the environment in cognitive development and mental processing.
This document discusses how today's youth, often referred to as "digital natives", interact with and learn through digital technologies in ways that are very different than previous generations. It notes that digital natives read blogs instead of newspapers, meet each other online first before in person, get their music online and illegally, and are more likely to communicate through instant messages than phone calls. Their social lives and civic activities are heavily mediated through digital tools. The document suggests educators need to understand these generational differences and how digital natives' skills developed through technologies could be better supported in education.
This report provides an overview of artificial intelligence including its goals, techniques, applications, and history. It defines AI as the science of creating intelligent machines and programs that mimic human intelligence. The report discusses how AI programming differs from traditional programming by being able to absorb new information without affecting its structure. It also outlines various AI techniques used to organize vast amounts of knowledge and several real-world applications of AI in areas like gaming, natural language processing, and robotics. Finally, the report compares human and artificial intelligence in terms of perception, memory, and problem-solving abilities.
Experience based Personal Multimedia Content Management SystemAdrian Hornsby
The document discusses personal content management through sensing and experience. It outlines moving from raw sensing to life sensing to gaining a continuum experience. This involves sensing data, analyzing patterns in behavior, social judgement and context to create living metadata. An experience-based ontology is proposed to organize multimedia content and close the loop between sensing, experience and personal content management.
Human Learning Online and Teaching Online Shalin Hai-Jew
Learners will…
consider how humans learn
review how humans learn online
study various types of online learning designs
review instructional design methods and standards
consider technical considerations in building online learning
explore various types of online learning designs
consider how online learning data may inform evolving learning designs
Cognitive computing aims to address complex problems characterized by ambiguity and uncertainty by learning from and interacting with people. The document discusses several topics related to cognitive computing and tacit knowledge, including:
- The history and traditions of cognitive science from the 1950s onward.
- How early experiments in the 1990s explored using computer-mediated telepresence to enable real-time collaborative creative work across distances, finding it can achieve similar results to physical presence.
- Questions around how cognitive computing systems today can help individuals and groups create value and preserve identity while working together.
- Whether true real-time collaborative creative work engaging tacit knowledge is possible digitally, or if physical presence is still needed.
The document discusses emerging technologies and their potential impacts and applications. It covers topics like artificial intelligence, computer vision, machine learning, internet of things, medical devices, generative design, zero-UI, emotion recognition, thermal cameras, predictive crime analysis, and digital profiling. Both benefits and risks are mentioned, such as streamlining design processes but also potential security vulnerabilities of connected devices. Overall, the document presents an overview of several rising technologies and considers their implications.
This document provides an overview of affective computing. It begins with definitions of affective computing and discusses its goals of allowing computers to recognize and express human emotions. Applications are described across various domains like education, banking, healthcare. Research areas like emotion detection from facial expressions, speech and other signals are outlined. Issues around privacy, misuse and effectiveness are noted. The document concludes that context awareness, multimodal detection and real-time processing will be important future research directions.
This document summarizes research on the Blue Eyes technology, which aims to give computers human-like perceptual abilities. It discusses how Blue Eyes uses non-intrusive sensors like video cameras and microphones to identify a user's actions, physical state, emotions, and where they are looking. This information is analyzed to build a model of the user over time to help the computer adapt and create a more productive environment. The document also reviews related work on detecting emotions from facial expressions and touch input and explores using eye tracking for computer input.
The document discusses the impact of technology on psychology and society. It covers how technology has transformed society and increased connectivity through social networking. It also discusses how gaming can impact cognition, noting that violent video games may briefly increase aggressiveness in children and regular exposure could make them "meaner". However, gaming may also improve visual attention and spatial reasoning skills. The document examines different perspectives on technology and looks at trends in technology use across generations.
The document discusses research into developing computers that can interact with humans more naturally by perceiving emotions and sensory inputs like humans. Specifically, it discusses:
1) The BLUE EYES technology which aims to give computers human-like perceptual and sensory abilities to understand facial expressions and emotional states.
2) An emotion mouse which measures physiological signals through a mouse to determine a user's emotional state and build a personalized model to help computers adapt to individual users.
3) Prior research linking facial expressions, physiological measurements like galvanic skin response and temperature, and emotional states which provide a framework for the emotion mouse research.
The document outlines a project conducted by Margie's Travel for the municipality of Eindhoven to increase data awareness on Stratumseind street. It involved analyzing the municipality's goals, conceptualizing solutions, and developing three concepts - WaveScape, Blob, and CityBeacon Eye. The concepts were presented to Living Lab Stratumseind and the municipality. The project helped the individuals at Margie's Travel strengthen their expertise in areas like user experience, technology, business, and creativity. It concluded that while the goals of raising awareness were met, further steps are needed to make IoT technologies more transparent and democratic on a societal level.
201812 design research on social cyber-physical systemsJun Hu
The document outlines research from the Department of Industrial Design on social cyber-physical systems, including projects called Social Bike, Social Car, Social Hue, Collective Stress, Strategy for Change, Heartbloom, and Systemic Change. The projects aim to enhance social connectivity and connectedness through applications like improving communication between drivers, social care for the elderly, and biofeedback to increase social awareness for children with heart disease.
The document provides advice on how to successfully complete a PhD at the TU/e. It discusses the requirements for a PhD degree, including completing an approved dissertation based on independent research that applies existing knowledge and contributes to the field. While publications are not required, it is recommended that each chapter be supported by 1 or more publications. Supervisors are expected to guide and support PhD students, while students are expected to conduct independent research and prepare for supervision meetings. Overall, the document emphasizes the importance of passion, preparation, hard work, and integrity in achieving a successful PhD.
The document provides information about TU/e (Eindhoven University of Technology) including that it is ranked in the top 100 of the QS World University Rankings and is a leading university in engineering, science and technology. It has 9 departments and offers undergraduate, master's and PhD programs. The master's programs include tracks in constructive design research, research design and development, and design leadership and entrepreneurship. The PhD and PDEng programs involve research and practical application respectively. Two specific PhD programs are described, one in design of systems with emerging technologies and one in user-system interaction.
Connectedness for enriching elderly care: Interactive Installation & System ...Jun Hu
Ageing has become a global topic with critical challenges for years. Currently, most attention of design and technological solutions for the ageing population is paid to physical health, mobility and safety, while in the field of social wellbeing and mental health, which are also important in ageing process, there is still much space to explore.
Closer to Nature: Interactive Systems for Seniors with Dementia in Long-term ...Jun Hu
People with dementia living in Long-term Care (LTC) are gradually experiencing diminished functional abilities caused by this brain disease. The declined cognitive functioning, decreased mobility, loss of memory and inner motivation provides inevitable challenges in engaging this group in activities. Lack of engagement are associated with disruptive behavioral and psychological symptoms of dementia (BPSD) such as agitation, wondering, apathy, passivity and depression. With no known cure in sight, developing and evaluating meaningful activities that foster and sustain engagement is critical for promoting quality of life for seniors with dementia in LTC.
This document discusses social applications of the Internet of Things. It defines the Internet of Things as connecting physical objects through standard internet protocols, allowing communication between things and people. The document provides examples of connected products that sense and share data through cloud services. It emphasizes designing for distributed intelligence, interaction and connectivity between physical objects, people and cloud services. The take home message is to think of the Internet of Things holistically and consider how to design for control, interaction and intelligence across connected people, things and networks.
This document outlines Jun Hu's vision for design research on social things. It discusses the Internet of Things and how things can be networked and connected to humans and each other through sensors, actuators and communication technologies. This forms the "Social Internet of Things". The document proposes that design research is needed to address new opportunities and challenges in designing networked products and adaptive, context-aware services. Key areas of focus include using social things to support cyber-physical systems, health/care, public spaces, and sustainability. The research methodology will intersect computational systems with social behavior using engineering, empirical and research through design methods. Education will be integrated by involving students in ongoing research and introducing basic skills through a makerspace.
Participatory Media Arts: A TU/e DESIS Lab projectJun Hu
This document describes a public media art installation called "Moon Rising from Sea" being designed for the center square of a new living and leisure area in Taicang, China. The installation consists of a 10x10m base with wall reliefs and an 8m tall structure resembling a sail with the moon rising from waves. Projectors will display images, animations and videos on the sail's surface in the evenings. The design process involved workshops to inspire designs from traditional arts. The installation will allow the public to contribute photos from social media for an interactive photo show, connecting people in Taicang to those overseas and reinstating Taicang's historical role as a port.
The document discusses the concepts of "interaction" and "fusion" in design. It argues that while interaction design focuses on the connections between individuals, the concept of "fusion design" aims to design for a state of symbiotic harmony where people, products, and environments blend together. The document suggests that Chinese traditional culture, which emphasizes that "you are among us and we are among you", provides a model for fusion design. Fusion design would focus on designing people, not just objects, to create a spirit of "great coordination" between humans and the systems they interact with. The concept of fusion design is just being introduced here without a full framework, but aims to spark discussion.
Traditional Dynamic Arts and Interaction DesignJun Hu
This document discusses design elements for interactive art installations for elderly patients with dementia. It explores how traditional performance elements like lighting, sound, movement can be adapted to encourage interaction. These elements aim to provide mental stimulation and feelings of happiness. Examples discussed include using music and light therapy to improve mood and cognition. The prototype installation brought joy to patients by allowing them to interact with glowing lights and curious sounds. Experts felt it prevented boredom and provided escape. Timing, script, and dividing the space into foreground, mid-ground and background can help capture attention. Adapting these performance elements may support meaningful social and cultural activities for dementia patients.
The document provides an overview of how to complete a PhD at the Eindhoven University of Technology. It discusses that a PhD involves conducting independent research that generates new knowledge and is documented in a dissertation. The dissertation must demonstrate solid research abilities and be approved by a PhD committee. While publications are not required, it is recommended to have 1 or more publications supporting each chapter. Supervisors guide PhD students and expect them to work independently while managing their own research. Overall, the document outlines the key requirements and expectations of a PhD and provides advice on successfully conducting research.
This document discusses academic publishing for designers. It covers several topics:
- Design can be viewed as both a craft and a science, with different approaches like design history, problem solving, and methodology.
- Quality is ensured through peer review, where experts judge papers and provide suggestions for improvement.
- There are risks of errors in peer review like rejecting quality papers or accepting weak papers.
- Rejected papers may eventually get published through alternative open access journals or repositories after multiple rejections elsewhere.
The document discusses the value of developing a "semiotic sensibility" in graphic and communication design students. It argues that semiotics, the study of signs and symbols, should be a fundamental part of design education rather than just a theoretical component. When applied over an extended period through various projects, semiotics can help students understand how design artifacts convey meaning and influence ideology. It also strengthens students' political awareness by providing contexts to question the effects of designs. Real-life collaborative projects further expose students to designing for social issues and help envision alternative design futures.
This document discusses the design of a nature-inspired relaxation space to help reduce work-related stress. It describes a GRIP service model that was used which emphasized the flexibility of design. The document outlines how designers conducted research on work-related stress by developing probes like personal balance, ambient experiences and paced breathing. It also discusses how prototypes were developed using principles of biophilia, biomimicry and life to support reducing stress. Finally, it notes that the GRIP approach brought together research, design, education and professionals.
Elements for Interaction Design in Public Spaces: Learning from Traditional D...Jun Hu
The document discusses elements for interaction design in public spaces, drawing from traditional dynamic arts. It discusses 4 elements: 1) Dramatic language according to different identities like operators, performers, and spectators. 2) Attention spaces with foreground, mid-ground, and background interactions. 3) Positions in space and time with front stage and back stage behaviors. 4) Elements from traditional dynamic art forms like words, sounds, signs, and symbols. The document aims to provide inspiration for interactive public art installations.
The document discusses the benefits of exercise for mental health. Regular physical activity can help reduce anxiety and depression and improve mood and cognitive functioning. Exercise causes chemical changes in the brain that may help boost feelings of calmness, happiness and focus.
Four students - Luca Giacolini, Bastiaan van Hout, Manon Junggeburth, and Tom van Rooij - were given an assignment by Mathias Funk, Joep Frens, and Bart Hengeveld to explore mechanisms for designing interactive musical systems. Luca Giacolini and Bastiaan van Hout presented their project Experio at the Desform Conference in September 2013, where they designed a system that transforms human movement in real time into music through experiential explorations.
This document discusses frameworks for understanding and designing new media experiences. It presents a FRAME model with five dimensions - Form, Reason, Audience, Material, and Extent - to characterize the design space. Two examples are provided to illustrate applying the framework: the Audiophoto Desk, which allows combining photos and audio into narratives; and the Collect Yourselves! system for live digital storytelling games. The talk seeks to understand how the design dimensions interact and how to evaluate what works for different experiences.
At a moment's notice, according to the pleasure of the holder the semiotics ...Jun Hu
This document appears to be notes from a lecture or research on various topics related to art, design, and social sciences. It references works by artists like Julian Rosefeldt and Alison Wilding. It also mentions different museums and exhibitions. Key areas of discussion include the semiotics of paper, Charles Sanders Peirce's graphical logic, and practice-led research in art and design. The document cites different authors and their works on related subjects.
GraphRAG for Life Science to increase LLM accuracyTomaz Bratanic
GraphRAG for life science domain, where you retriever information from biomedical knowledge graphs using LLMs to increase the accuracy and performance of generated answers
Ocean lotus Threat actors project by John Sitima 2024 (1).pptxSitimaJohn
Ocean Lotus cyber threat actors represent a sophisticated, persistent, and politically motivated group that poses a significant risk to organizations and individuals in the Southeast Asian region. Their continuous evolution and adaptability underscore the need for robust cybersecurity measures and international cooperation to identify and mitigate the threats posed by such advanced persistent threat groups.
Building Production Ready Search Pipelines with Spark and MilvusZilliz
Spark is the widely used ETL tool for processing, indexing and ingesting data to serving stack for search. Milvus is the production-ready open-source vector database. In this talk we will show how to use Spark to process unstructured data to extract vector representations, and push the vectors to Milvus vector database for search serving.
Ivanti’s Patch Tuesday breakdown goes beyond patching your applications and brings you the intelligence and guidance needed to prioritize where to focus your attention first. Catch early analysis on our Ivanti blog, then join industry expert Chris Goettl for the Patch Tuesday Webinar Event. There we’ll do a deep dive into each of the bulletins and give guidance on the risks associated with the newly-identified vulnerabilities.
Trusted Execution Environment for Decentralized Process MiningLucaBarbaro3
Presentation of the paper "Trusted Execution Environment for Decentralized Process Mining" given during the CAiSE 2024 Conference in Cyprus on June 7, 2024.
TrustArc Webinar - 2024 Global Privacy SurveyTrustArc
How does your privacy program stack up against your peers? What challenges are privacy teams tackling and prioritizing in 2024?
In the fifth annual Global Privacy Benchmarks Survey, we asked over 1,800 global privacy professionals and business executives to share their perspectives on the current state of privacy inside and outside of their organizations. This year’s report focused on emerging areas of importance for privacy and compliance professionals, including considerations and implications of Artificial Intelligence (AI) technologies, building brand trust, and different approaches for achieving higher privacy competence scores.
See how organizational priorities and strategic approaches to data security and privacy are evolving around the globe.
This webinar will review:
- The top 10 privacy insights from the fifth annual Global Privacy Benchmarks Survey
- The top challenges for privacy leaders, practitioners, and organizations in 2024
- Key themes to consider in developing and maintaining your privacy program
How to Interpret Trends in the Kalyan Rajdhani Mix Chart.pdfChart Kalyan
A Mix Chart displays historical data of numbers in a graphical or tabular form. The Kalyan Rajdhani Mix Chart specifically shows the results of a sequence of numbers over different periods.
Letter and Document Automation for Bonterra Impact Management (fka Social Sol...Jeffrey Haguewood
Sidekick Solutions uses Bonterra Impact Management (fka Social Solutions Apricot) and automation solutions to integrate data for business workflows.
We believe integration and automation are essential to user experience and the promise of efficient work through technology. Automation is the critical ingredient to realizing that full vision. We develop integration products and services for Bonterra Case Management software to support the deployment of automations for a variety of use cases.
This video focuses on automated letter generation for Bonterra Impact Management using Google Workspace or Microsoft 365.
Interested in deploying letter generation automations for Bonterra Impact Management? Contact us at sales@sidekicksolutionsllc.com to discuss next steps.
Skybuffer SAM4U tool for SAP license adoptionTatiana Kojar
Manage and optimize your license adoption and consumption with SAM4U, an SAP free customer software asset management tool.
SAM4U, an SAP complimentary software asset management tool for customers, delivers a detailed and well-structured overview of license inventory and usage with a user-friendly interface. We offer a hosted, cost-effective, and performance-optimized SAM4U setup in the Skybuffer Cloud environment. You retain ownership of the system and data, while we manage the ABAP 7.58 infrastructure, ensuring fixed Total Cost of Ownership (TCO) and exceptional services through the SAP Fiori interface.
Generating privacy-protected synthetic data using Secludy and MilvusZilliz
During this demo, the founders of Secludy will demonstrate how their system utilizes Milvus to store and manipulate embeddings for generating privacy-protected synthetic data. Their approach not only maintains the confidentiality of the original data but also enhances the utility and scalability of LLMs under privacy constraints. Attendees, including machine learning engineers, data scientists, and data managers, will witness first-hand how Secludy's integration with Milvus empowers organizations to harness the power of LLMs securely and efficiently.
Have you ever been confused by the myriad of choices offered by AWS for hosting a website or an API?
Lambda, Elastic Beanstalk, Lightsail, Amplify, S3 (and more!) can each host websites + APIs. But which one should we choose?
Which one is cheapest? Which one is fastest? Which one will scale to meet our needs?
Join me in this session as we dive into each AWS hosting service to determine which one is best for your scenario and explain why!
Salesforce Integration for Bonterra Impact Management (fka Social Solutions A...Jeffrey Haguewood
Sidekick Solutions uses Bonterra Impact Management (fka Social Solutions Apricot) and automation solutions to integrate data for business workflows.
We believe integration and automation are essential to user experience and the promise of efficient work through technology. Automation is the critical ingredient to realizing that full vision. We develop integration products and services for Bonterra Case Management software to support the deployment of automations for a variety of use cases.
This video focuses on integration of Salesforce with Bonterra Impact Management.
Interested in deploying an integration with Salesforce for Bonterra Impact Management? Contact us at sales@sidekicksolutionsllc.com to discuss next steps.
Unlock the Future of Search with MongoDB Atlas_ Vector Search Unleashed.pdfMalak Abu Hammad
Discover how MongoDB Atlas and vector search technology can revolutionize your application's search capabilities. This comprehensive presentation covers:
* What is Vector Search?
* Importance and benefits of vector search
* Practical use cases across various industries
* Step-by-step implementation guide
* Live demos with code snippets
* Enhancing LLM capabilities with vector search
* Best practices and optimization strategies
Perfect for developers, AI enthusiasts, and tech leaders. Learn how to leverage MongoDB Atlas to deliver highly relevant, context-aware search results, transforming your data retrieval process. Stay ahead in tech innovation and maximize the potential of your applications.
#MongoDB #VectorSearch #AI #SemanticSearch #TechInnovation #DataScience #LLM #MachineLearning #SearchTechnology
Unlock the Future of Search with MongoDB Atlas_ Vector Search Unleashed.pdf
Baarlo nijholt
1. 7/15/2013
1
Human Sensing, Tagging, and
Interaction
Anton Nijholt
Human Media Interaction
University of Twente
SETTING THE STAGE
Part I
Human Sensing, Tagging, and Interaction
2. 7/15/2013
2
IR & Interaction
Human Sensing in Sensor Equipped Environments
- Human (Interaction) Behavior
- Human Activities
Human Sensing to Support (in Real-time) Human
Activities
- Requires some (Real-time) Understanding of Human Activities
Human Sensing to (Automatically) Tag Human Activities
for Future Retrieval
- Requires some Understanding of Human Activities
Convergence of Research on Retrieval and
Understanding of Human Activity
Human Sensing, Tagging, and Interaction
IR & Interaction: Developments
– Sensing (registration)
text, pictures, audio, video, …
activity (proximity, movements, …)
environment (temperature, humidity, weather, …)
human (neuro-)physiological information (heart rate
variability, blood pressure, skin conductivity, brain activity, …)
– Tagging (from annotating to interpretation)
off-line, on-line (real-time)
manual, semi-automatic, fully automatic, …
– Interaction
ask, interpret, act, …
feedback (adapt/filter/…)
conversation, question-answering, dialogue
Human Sensing, Tagging, and Interaction
3. 7/15/2013
3
Sensing, Tagging, Interaction
Sensing
– Cameras, microphones, keyboard, mouse,
joystick, physiological, proximity, pressure, EEG,
…
Tagging
– From annotation to interpretation
– From manual to semi-automatic to fully automatic
Interaction
– (Real-time) fully automatic interpretation
Human Sensing, Tagging, and Interaction
Sensor equipped environments
Sensor equipped inhabitants/visitors
Humans, virtual humans, (humanoid) robots,
pets, ‘living’ furniture
Displays everywhere, access from/to outside
worlds (physical, virtual)
Future Internet: ‘Internet of Things’
Smart Environments
Human Sensing, Tagging, and Interaction
4. 7/15/2013
4
Asking in Smart Environments
Ask questions in smart environments
Get answers in smart environments
Requires understanding of what’s going on
and has been going on in an environment
What kinds of questions? What ways are
there to answer questions?
Human Sensing, Tagging, and Interaction
Possibility of continuously capturing, analyzing,
interpreting and storing of temporal streams of
data that concern a particular person
Tagging (or description) of everyday life events
in order to allow future retrieval of events or the
experiences related to events
This allows someone to build her own digital
memory of (potentially) interesting events
‘Electronic Chronicles’, ‘Memories for Life’, ‘Lifelogs’,
‘Digital Prosthetic Memories’
Towards Personal Memories
Human Sensing, Tagging, and Interaction
5. 7/15/2013
5
What Kinds of Questions?
One fool may ask more than seven wise
men can answer.
[1666 G. Torriano]
Ask about events, human behavior,
interactions, feelings, experiences, … in our
smart environments and our (artificial)
personal memories
Human Sensing, Tagging, and Interaction
Retrieval of events
– To use, to show
– To re-experience
Retrieval of Experiences
– By retrieving (similar) events
– By retrieving/regenerating the physiological
experiences
Retrieval of Thoughts/Mind States
Retrieval of Events and
Experiences
Human Sensing, Tagging, and Interaction
6. 7/15/2013
6
“What was the name of the person I met
yesterday evening?”
“How did the strawberry jam I bought last week
disappear in 4 days?”
“Why did my girlfriend leave me?”
“Why do I feel sad?”
Retrieval of Events and
Experiences
Human Sensing, Tagging, and Interaction
Retrieval Systems?
– Google, Yahoo, …
Chat Bots?
– Eliza, Alice, Yabberwocky, …
Question-Answering Systems
Dialogue Systems
– Travel information systems, call-centers, …
Who to Ask? More Traditionally
Human Sensing, Tagging, and Interaction
8. 7/15/2013
8
ELIZA (1966)
You: I've had a terrible day
Eliza: Please go on.
You: First we were under attack
Eliza: Please go on.
You: Then our vehicle struck a landmine
Eliza: Please go on.
You: Then my friend got killed
Eliza: Why do you say your friend got killed?
Asking for Sharing of Feelings
Human Sensing, Tagging, and Interaction
ALICE
Human: I've had a terrible day
ALICE: What happened to it?
Human: First we were under attack
ALICE: And then what happened? And
second?
Human: Then our vehicle struck a landmine
ALICE: Fascinating. It would be nice if our
vehicle struck a landmine.
Human: Then my friend got killed
ALICE: Good reasoning. How well do you
know this person
Asking for Sharing of Feelings
Human Sensing, Tagging, and Interaction
9. 7/15/2013
9
Multi-modal Question-Answering
Human Sensing, Tagging, and Interaction
Multi-modal Question-Answering
– Sensing (registration)
text, pictures, audio, video, … ; activity (proximity, movements, …);
environment (temperature, humidity, weather, …); human (neuro-
)physiological information (heart rate variability, blood pressure, skin
conductivity, brain activity, …)
– Tagging (from annotating to interpretation)
off-line, on-line on-going activities; manual, semi-automatic, fully
automatic, …
– Interaction
ask, interpret, act, …; feedback (adapt/filter/influence…)
interpretation of user feedback and providing more information
Q&A about (real-time or past) activity?
Human Sensing, Tagging, and Interaction
10. 7/15/2013
10
Dialogue Systems
Far from Natural
Amtrak Julie
Human Sensing, Tagging, and Interaction
Multi-modal Dialogue Systems
Conversation
Negotiation
Informative dialogue
– Speech recognition, natural language processing,
gesture processing, facial expression processing,
(neuro-)physiological signal processing, ….
– User modeling, common-sense/world/domain
knowledge modeling, interaction modeling, …
Human Sensing, Tagging, and Interaction
11. 7/15/2013
11
Gap
Large gap between what existing systems can offer for
retrieval and interaction and what can be sensed and
collected
Collect more detailed data, collect other data?
Look at other ways to disseminate information? Predict
interest, provide easy access, and make information
browsable?
Tools and environments to provide answers to specific
questions; no ‘free’ conversation, Q&A, or dialogue
Human Sensing, Tagging, and Interaction
Back on the Track: Interest
‘Tagging’ of humans or human activity
(including human-human, human-system,
and multi-party interaction)
Tagging
– From low-level (‘counting’) to high-level
(interpretation)
– From manual to semi-automatic to automatic
From off-line retrieval to real-time interaction
and support
Human Sensing, Tagging, and Interaction
12. 7/15/2013
12
Aims (1)
Collecting information about humans and
human behavior (sensing) allows us to:
– understand them (their questions, their needs,
their behavior)
– provide real-time support, also by anticipating
their needs and pro-actively support them
– allow understanding (hence, support) and
retrieval of ‘human information’, ‘human-human
interaction information’, ‘multi-party interaction
information’, events, experiences, ….
Human Sensing, Tagging, and Interaction
Aims (2)
People as ‘Content’
– observe content, learn about content, store
content, …
– process content, interpret content, transform
content, mediate content, retrieve content, …
– interact with content, interact with a virtual
user/partner or virtual users/partners,
representing (maybe not always) ‘real’
users/partners
Make humans computable
Human Sensing, Tagging, and Interaction
14. 7/15/2013
14
Sensing People
OTHERS?
(1) Keyboard, Mouse, Joystick,
Balance board, Wiimote, Nunchuck,
3D Mouse, Tangibles, ….
(2) Sensors: Proximity, Pressure,
(3) Wearables, mobile devices
(location-based social networks such
as FourSquare), …..
Physiological: skin conductivity,
heart rate (variability), blood
pressure, …
Brain Imaging: regions, functions,
methods (EEG, fNIRS, …)
Human Sensing, Tagging, and Interaction
Sensing People
Wireless headsets
Human Sensing, Tagging, and Interaction
Implants
15. 7/15/2013
15
meeting environments,
home & office
environments, research
teams, healthcare,
education, sports,
training, games,
entertainment, ….
Pro-active and Reactive Support
‘Daily Life’ Interactions
Human Sensing, Tagging, and Interaction
Social Robots
Virtual Humans
Environments
Behavior Interpretation & Behavior Generation
Human Behavior Generation
Human Sensing, Tagging, and Interaction
… and replay/manipulate situations in VR ..
16. 7/15/2013
16
‘Daily Life’ Interactions
Example of
Human-Human
Interaction
Human Sensing, Tagging, and Interaction
Grice (1975)
Many others
– Searle (1975): indirect speech acts
– Leech (1983): maxims of politeness
– Clark & Wilkes-Gibbs (1986): principle of ‘least
collaborative effort’ to guide grounding, accepting
referring expressions
– Grosz & Sidner (1990): shared plans
– Cohen & Levesque (1991): joint intentions
Listening to Interaction
Interaction always Requires (some
meta-level) Cooperation
Human Sensing, Tagging, and Interaction
17. 7/15/2013
17
Nonverbal Interaction
Looking at Interaction
Human Sensing, Tagging, and Interaction
Lots of ‘Non-Cooperative’ (verbal/nonverbal)
behavior
– Not always telling the truth (can be a social lubricant)
– Self-interest, exaggerated/false politeness
– Teasing, provoking, joking, flirting, . . .
– Play different roles for different audiences
– Mediated interaction: chatting, instant messaging, twittering, …
– Games, sports, education, …
Human-Human Interaction
‘Daily Life’ Interactions
Human Sensing, Tagging, and Interaction
19. 7/15/2013
19
Sensor Equipped Environments
Observe verbal and nonverbal interaction
behavior
– Human-environment/devices/ …. interaction
behavior
– Human-human interaction behavior
– Multi-party interaction behavior
Human Sensing, Tagging, and Interaction
‘CONSTRAINED’ INTERACTION
ENVIRONMENTS
Human Sensing, Tagging, and Interaction
20. 7/15/2013
20
Application Scenarios
– Retrieve/look up information on previous meetings
– Audit unattended meetings
– Reminders about a previous meeting during a
meeting
– Catching up on a meeting in progress
– Detect meeting failures, redundancy, conflict,
dominance, etc.
AMI Meeting Project
Speech recognition
Speaker localization and identification
Gesture recognition and tracking
Emotion recognition
Event and topic segmentation
Argumentative structure
Content analysis
Summaries of meetings/events
Processing Technologies
Human Sensing, Tagging, and Interaction
21. 7/15/2013
21
Manual and (semi-)
automatic annotating
Audio-visual analysis
(speech processing,
computer vision)
Gaze, gestures, posture,
head orientation, facial
expression, prosody, …
Tracking, identification.
emotion detection, turn
taking, addressee
detection, …
… listening …
Technologies for ….
Human Sensing, Tagging, and Interaction
What topics are discussed and when?
What decisions are made and by whom?
What roles do the participants play?
Cooperative / non-cooperative partners?
Covergence to agreement, disagreement?
What positions do they take on issues?
What activities are completed?
What tasks are assigned or reported done?
Questions to be Answered
22. 7/15/2013
22
Instrumented meeting rooms at IDIAP, UEDIN and TNO
Off-line access to
captured meetings
Real-time support to
Meeting Participants
Sensing and Interpreting
everything that is
important to the Meeting
and providing Reactive
and Proactive Support
Corpus Collection
Looking at Interaction
Change of Aims
Human Sensing, Tagging, and Interaction
Meeting environment needs to understand verbal and nonverbal behavior of its inhabitants
EU FP6 AMI & AMIDA Projects
Pro-active and Reactive
Environments
Looking at Interaction
Human Sensing, Tagging, and Interaction
23. 7/15/2013
23
Annotation,
Analysis,
Heuristics,
Models
Understand &
Generate
FP6: AMIDA
FP7: Humaine
FP7: SEMAINE
FP7: SSPNet
Looking at Interaction
Human Sensing, Tagging, and Interaction
accept agree angry astounded attentive believe bored
compassionate considering disagree disappointed disbelieve
disdain disgust dislike distrust doubt encourage helpless
interested like meaningless not interested oh no not again pity
pondering refuse sad sorrow surprised thinking thoughtful
uncertain understand unhappy worried not understand
Looking at Interaction
Human Sensing, Tagging, and Interaction
24. 7/15/2013
24
Dialogue acts
Gaze direction
Addressee
Affective state
Argumentation
Turn taking
……
Annotation Tools
Human Sensing, Tagging, and Interaction
Manual and (semi-)
automatic annotating
Audio-visual analysis
(speech processing,
computer vision)
Gaze, gestures, posture,
head orientation, facial
expression, prosody, …
Tracking, identification.
emotion detection, turn
taking, addressee
detection, …
… listening …
Processing Technologies
Human Sensing, Tagging, and Interaction
25. 7/15/2013
25
Head
Nod/Shake
Detector
Hand Raise
Detector
Processing Technologies
Human Sensing, Tagging, and Interaction
Methodology
• Analysis of annotations
brings us heuristics, rules
and models
• Annotations are starting
point for machine learning
of rules and models
• Rules and models become
algorithms that allow
interpretation and adequate
reactions (re-active and
pro-active) on demands
and events
Heuristics & Models
Human Sensing, Tagging, and Interaction
27. 7/15/2013
27
Looking at Interaction
Behavior coordination lets interactants
assimilate their behaviors in form, content or
timing;
Belief coordination leads to compatible
knowledge about specific topics, tasks, or
each other;
Attitude coordination regulates the
individual’s stances toward each other or
external objects.
Stefan Kopp, 2010
Human Sensing, Tagging, and Interaction
Looking at Interaction
Human Sensing, Tagging, and Interaction
28. 7/15/2013
28
Looking at Interaction
Recent Research: Mimicry Analysis
Joint research
with Imperial
College
Human Sensing, Tagging, and Interaction
‘Daily Life’: Looking at Interaction
Joint research
with Imperial
College
Recent Research: Mimicry Analysis
Human Sensing, Tagging, and Interaction
29. 7/15/2013
29
Example Applications
SAL
Poppy
Spike
Prudence
Obadiah
Human Sensing, Tagging, and Interaction
SEMAINE
Summary:
Sensing & Interpreting Interaction
Understand interaction & collaboration
behavior
Provide implicit interaction/real-time support
Provide/generate natural ‘machine’ behavior
(virtual humans, social robots, avatars in
games, interfaces
Towards retrieval of events and experiences
Human Sensing, Tagging, and Interaction