Lecture Notes for Lecture 11

1. ICS3211 - Intelligent
Interfaces II
Combining design with technology for effective human-
computer interaction
Week 11 Department of AI,
University of Malta

2. Affective Computing for
Intelligent Interfaces
Week 11 overview:
• An Intro to Affective Computing
• Affective Computing for Intelligent Interfaces

3. Affective Computing
• Affective computing is a
multidisciplinary field that
focuses on the study and
development of systems
and devices capable of
recognising, interpreting,
processing, and simulating
human emotions or affect.
It combines computer
science, psychology, and
cognitive science to create
more empathetic and
human-like interactions
between humans and
machines.

4. Affective Computing

5. Benefits of Affective
Computing
• Personalised and intuitive interactions
• Improved Decision Making
• Enhanced Patient Care
• Immersive Experiences
• Safer Driving
• Improved Work Conditions

6. Challenges of Affective
Computing
• Sensing and Recognising Emotions
• Affect Modelling
• Emotion Expression
• Ethics
• Data Protection
• Interoperability
• Utility of Considering Affect in HCI
• Adaptation to Individual Differences

7. Discussion
• Working in groups engage in a critical discussion
about the ethical implications of affective computing,
as pioneered by Rosalind Picard and others in the
field. Affective computing, the study and development
of systems and devices that can recognise, interpret,
process, and simulate human emotions, raises several
ethical questions, particularly as this technology
becomes more integrated into various aspects of daily
life. How does affective computing, combined with
immersive realities, impinge on the individuals and
what can be done to mitigate risks of individual harms.

8. Emotion Recognition

9. Facial Expression Analysis
• Facial Changes in response to a person’s
internal states:
• Face Acquisition
• Facial Data Extraction
• Facial Expression Recognition
• CNNs and SVMs used to classify expressions
corresponding to emotions

10. Ethics for Facial Expression
Analysis
• EU AI Act - 1st World Comprehensive AI
Regulation
• Limit the use of biometric identification
systems
• Ban on the use of FRT for emotion recognition
at the workplace, untargeted scraping of facial
images from Internet, or CCTV to create facial
recognition datasets

11. Voice Tone Analysis
• Examination of Vocal Characteristics and
patterns in spoken communication:
• Pitch
• Tone
• Speech rate
• Amazon Chime’s SDK uses DNN to estimate
probabilities that speech segment expresses
positive, negative or neutral sentiment.

12. Physiological Signal
Processing
• Monitors Body Functions and Measures Impacts
of Factors on Physiological Functions
• Wearable sensors often used for ML
techniques

13. Case Studies in Affective
Computing
• Healthcare
• Retail and Marketing
• Customer Service
• Education
• Human Resources
• Autonomous Driving
• Gaming
https://
research.aimultiple.com/
emotional-ai-examples/
https://builtin.com/artificial-
intelligence/emotion-ai
https://
www.xenonstack.com/blog/
emotional-ai

14. Task
• Work in small groups to design a basic outline for
an emotion-aware interface for a specific
application (e.g., healthcare, education, customer
service).
• Conceptualise how affective computing can be
integrated into an intelligent interface and to
address potential challenges.