Emotion Recognition in Artificial Intelligence by Valentina Franzoni, Ph.D. in Engineering for Computer Science, Research Fellow in Artificial Intelligence, adjunct professor in Operative Systems
“Emotion Recognition in Artificial Intelligence” by Valentina Franzoni, Ph.D. in Engineering for Computer Science, Research Fellow in Artificial Intelligence, adjunct professor in Operative Systems
Abstract : Affective computing and social robotics study Emotion Recognition to improve the emotional interaction between humans and machines, but also how to design cognitive architectures which included biomimetic elements related to emotions. It is difficult to find any field related to robotics and AI which is not directly or indirectly related to the implementation of emotional values. This revolution is not only about formal heuristics, or the called "algorithmic society", but is also explicitly related to the creation of strong interactions between humans and machines, considering roles and ethics. The complex problem needs a multidisciplinary approach, ranging from computer science to psychology, neurology, and life sciences.
As electricity is difficult to store, it is crucial to strictly maintain the balance between production and consumption. The integration of intermittent renewable energies into the production mix has made the management of the balance more complex. However, access to near real-time data and communication with consumers via smart meters suggest demand response. Specifically, sending signals would encourage users to adjust their consumption according to the production of electricity. The algorithms used to select these signals must learn consumer reactions and optimize them while balancing exploration and exploitation. Various sequential or reinforcement learning approaches are being considered.
Online violence amplifies IRL discriminations, and the lack of diversity grows in a vicious circle. Understanding cyber-violence, its forms and mechanisms, can help us fight back. To process massive volumes of data, AI finally comes into play for good.
In the energy sector, the use of temporal data stands as a pivotal topic. At GRDF, we have developed several methods to effectively handle such data. This presentation will specifically delve into our approaches for anomaly detection and data imputation within time series, leveraging transformers and adversarial training techniques.
Natasha shares her experience to delve into the complexities, challenges, and strategies associated with effectively leading tech teams dispersed across borders.
Nour and Maria present the work they did at Tweag, Modus Create innovation arm, where the GenAI team developed an evaluation framework for Retrieval-Augmented Generation (RAG) systems. RAG systems provide an easy and low-cost way to extend the knowledge of Large Language Models (LLMs) but measuring their performance is not an easy task.
The presentation will review existing evaluation frameworks, ranging from those based on the traditional ML approach of using groundtruth datasets, including Tweag's, to those that use LLMs to compute evaluation metrics.
It will also delve into the practical implementation of Tweag's chatbot over two distinct documents datasets and provide insights on chunking, embedding and how open source and commercial LLMs compare.
Sharone Dayan, Machine Learning Engineer and Daria Stefic, Data Scientist, both from Contentsquare, delve into evaluation strategies for dealing with partially labelled or unlabelled data.
As electricity is difficult to store, it is crucial to strictly maintain the balance between production and consumption. The integration of intermittent renewable energies into the production mix has made the management of the balance more complex. However, access to near real-time data and communication with consumers via smart meters suggest demand response. Specifically, sending signals would encourage users to adjust their consumption according to the production of electricity. The algorithms used to select these signals must learn consumer reactions and optimize them while balancing exploration and exploitation. Various sequential or reinforcement learning approaches are being considered.
Online violence amplifies IRL discriminations, and the lack of diversity grows in a vicious circle. Understanding cyber-violence, its forms and mechanisms, can help us fight back. To process massive volumes of data, AI finally comes into play for good.
In the energy sector, the use of temporal data stands as a pivotal topic. At GRDF, we have developed several methods to effectively handle such data. This presentation will specifically delve into our approaches for anomaly detection and data imputation within time series, leveraging transformers and adversarial training techniques.
Natasha shares her experience to delve into the complexities, challenges, and strategies associated with effectively leading tech teams dispersed across borders.
Nour and Maria present the work they did at Tweag, Modus Create innovation arm, where the GenAI team developed an evaluation framework for Retrieval-Augmented Generation (RAG) systems. RAG systems provide an easy and low-cost way to extend the knowledge of Large Language Models (LLMs) but measuring their performance is not an easy task.
The presentation will review existing evaluation frameworks, ranging from those based on the traditional ML approach of using groundtruth datasets, including Tweag's, to those that use LLMs to compute evaluation metrics.
It will also delve into the practical implementation of Tweag's chatbot over two distinct documents datasets and provide insights on chunking, embedding and how open source and commercial LLMs compare.
Sharone Dayan, Machine Learning Engineer and Daria Stefic, Data Scientist, both from Contentsquare, delve into evaluation strategies for dealing with partially labelled or unlabelled data.
Laure talked about a very hot topic in the community at the moment with the ChatGPT phenomenon: how to supervise a PhD thesis in NLP in the age of Large Language Models (LLMs)?
Abstract: Who hasn't heard of the "Pilot Syndrome"? 85% of Data Science Pilots remain pilots and do not make it to the production stage. Let's build a production-ready and end-user-friendly Data Science application. 100% python and 100% open source.
Phase 1 | Building the GUI: create an interactive and powerful interface in a few lines of code
Phase 2 | Integrated back end: Manage your models and pipelines and create scenarios the smart way
"Nature Language Processing for proteins" by Amélie Héliou, Software Engineer @ Google Research
Abstract: Over the past few months, Large Language Models have become very popular.
We'll see how a simple LLM works, from input sentence to prediction.
I'll then present an application of LLM to protein name prediction.
Twitter: @Amelie_hel
"We are not passing by, and we are not a trend". What if an automated and large scale version of the Bechdel-Wallace test could confirm the speech of Alice Diop at the Cesar 2023?
That's the objective of BechdelAI : to build a tool based on Artificial Intelligence and open-source, allowing to measure the inequalities and the under-representation of women in movies and audiovisual.
"Emergency plan to secure winter: what are the measures set up by RTE?" by Sophie Diakhate, Ingénieure Génie électrique, Consultante en énergie et utilities at Yélé
Abstract: The french electric system is currently going through an exceptional crisis, threatening the electric supplies for this winter, and potentially the next ones. As the guarantor of the balance between supply and demand, RTE must assume the security of supply security in France. They set up an emergency plan to secure winters. Yélé helps RTE to carry on that plan.
We are going to see what are the primary measures proposed by RTE for the winter 2022-2023, and the options for individuals and the industry to reduce the risks of network load shedding.
"New edge prediction and anomaly-detection in large computer networks" by Dr Silvia Metelli, Marie Skołodowska-Curie Individual Fellow
Abstract : Monitoring computer network traffic in search for anomalous behaviour is both a challenging and important task for cyber-security. New edges, i.e. connections from a host or user to a computer that has not been connected to before, provide potentially strong statistical evidence for detecting anomalies and in rare cases might suggest the presence of intruders or malicious activity. In this talk, I will introduce a robust Bayesian model and anomaly detection method for simultaneously characterising network structure and modelling likely new edge formation in a large computer network graph. What constitutes normal behaviour for some hosts might be very unusual for some others and thus examining existing network structure (e.g. clusters of similar clients and servers) is key for accurately predicting likely future interactions. Finally, the model is used to construct an anomaly detection method, which successfully identifies some of the machines known to be compromised when demonstrated on real computer network authentication data.
Interest has increased in the use of prognosis factors as a cursor for breast cancer personalized treatment. For clinicians, early detection of those factors can be helpful for a good management of the disease and for the choice of an efficient treatment. Moreover, it exists a huge amount of meaningful information in pathological reports, biological measurements and clinical information in a patient journey that remain unexploited. In that context, I propose to develop and apply novel machine learning techniques to predict cancer outcome such as recurrence or survival from multi-modal breast cancer patient data (including medical notes in natural languages and the outcome of various lab analyses). For that, I use a deep neural sequence transduction for electronic health records called BEHRT1. This model is inspired from one of the most powerful transformer-based architecture in Natural Language Processing: BERT2.
During the joint meetup with Duchess France and PyLadies Paris, Deborah Boyenval, PhD Student at Université Côté d'Azur presented a part of her PhD work: “Formal modeling of biological cyclic behavior with control points: the case of the cell cycle”.
The main limitation of biologists rooted in an experimental practice is the ability to perform rigorous proofs in the absence of a language for formalizing the biological knowledge extracted from their experiments.
Biologists have identified numerous biochemical and genetic mechanisms involved in physiological functions or diseases, but once this knowledge is linked together it remains extremely difficult and expensive to predict the impact of genetic mechanisms on physiological functions.
Déborah will present to us her thesis, which focuses firstly on a reasonable mathematical specification of complex biological functions such as cell cycle checkpoints, which represent the main barrier against cancer. Secondly, she focused on the development of an automated proof method, using the mentioned tools, proving whether a set of genetic regulations is sufficient to generate cell cycle checkpoints.
A look behind L’Oréal’s tool for consumer feedback analysis. We will discuss how NLP and Computer Vision have been applied to analyse large volumes of product reviews. Specifically, we’ll talk about topic extraction, sentiment analysis and topic enrichment in NLP, and siamese neural networks with triplet loss in Computer Vision.
"Blood flow simulation for clinical applications" by Dr Irene Vignon-Clementel, Directrice de recherche @Inria
Abstract : The dynamics of how blood flows into our body can be numerically simulated. Such simulations provide an 'augmented intelligence' to better understand cardiovascular and organ disease and plan their treatment.
Sachpazis:Terzaghi Bearing Capacity Estimation in simple terms with Calculati...Dr.Costas Sachpazis
Terzaghi's soil bearing capacity theory, developed by Karl Terzaghi, is a fundamental principle in geotechnical engineering used to determine the bearing capacity of shallow foundations. This theory provides a method to calculate the ultimate bearing capacity of soil, which is the maximum load per unit area that the soil can support without undergoing shear failure. The Calculation HTML Code included.
Cosmetic shop management system project report.pdfKamal Acharya
Buying new cosmetic products is difficult. It can even be scary for those who have sensitive skin and are prone to skin trouble. The information needed to alleviate this problem is on the back of each product, but it's thought to interpret those ingredient lists unless you have a background in chemistry.
Instead of buying and hoping for the best, we can use data science to help us predict which products may be good fits for us. It includes various function programs to do the above mentioned tasks.
Data file handling has been effectively used in the program.
The automated cosmetic shop management system should deal with the automation of general workflow and administration process of the shop. The main processes of the system focus on customer's request where the system is able to search the most appropriate products and deliver it to the customers. It should help the employees to quickly identify the list of cosmetic product that have reached the minimum quantity and also keep a track of expired date for each cosmetic product. It should help the employees to find the rack number in which the product is placed.It is also Faster and more efficient way.
More Related Content
More from Paris Women in Machine Learning and Data Science
Laure talked about a very hot topic in the community at the moment with the ChatGPT phenomenon: how to supervise a PhD thesis in NLP in the age of Large Language Models (LLMs)?
Abstract: Who hasn't heard of the "Pilot Syndrome"? 85% of Data Science Pilots remain pilots and do not make it to the production stage. Let's build a production-ready and end-user-friendly Data Science application. 100% python and 100% open source.
Phase 1 | Building the GUI: create an interactive and powerful interface in a few lines of code
Phase 2 | Integrated back end: Manage your models and pipelines and create scenarios the smart way
"Nature Language Processing for proteins" by Amélie Héliou, Software Engineer @ Google Research
Abstract: Over the past few months, Large Language Models have become very popular.
We'll see how a simple LLM works, from input sentence to prediction.
I'll then present an application of LLM to protein name prediction.
Twitter: @Amelie_hel
"We are not passing by, and we are not a trend". What if an automated and large scale version of the Bechdel-Wallace test could confirm the speech of Alice Diop at the Cesar 2023?
That's the objective of BechdelAI : to build a tool based on Artificial Intelligence and open-source, allowing to measure the inequalities and the under-representation of women in movies and audiovisual.
"Emergency plan to secure winter: what are the measures set up by RTE?" by Sophie Diakhate, Ingénieure Génie électrique, Consultante en énergie et utilities at Yélé
Abstract: The french electric system is currently going through an exceptional crisis, threatening the electric supplies for this winter, and potentially the next ones. As the guarantor of the balance between supply and demand, RTE must assume the security of supply security in France. They set up an emergency plan to secure winters. Yélé helps RTE to carry on that plan.
We are going to see what are the primary measures proposed by RTE for the winter 2022-2023, and the options for individuals and the industry to reduce the risks of network load shedding.
"New edge prediction and anomaly-detection in large computer networks" by Dr Silvia Metelli, Marie Skołodowska-Curie Individual Fellow
Abstract : Monitoring computer network traffic in search for anomalous behaviour is both a challenging and important task for cyber-security. New edges, i.e. connections from a host or user to a computer that has not been connected to before, provide potentially strong statistical evidence for detecting anomalies and in rare cases might suggest the presence of intruders or malicious activity. In this talk, I will introduce a robust Bayesian model and anomaly detection method for simultaneously characterising network structure and modelling likely new edge formation in a large computer network graph. What constitutes normal behaviour for some hosts might be very unusual for some others and thus examining existing network structure (e.g. clusters of similar clients and servers) is key for accurately predicting likely future interactions. Finally, the model is used to construct an anomaly detection method, which successfully identifies some of the machines known to be compromised when demonstrated on real computer network authentication data.
Interest has increased in the use of prognosis factors as a cursor for breast cancer personalized treatment. For clinicians, early detection of those factors can be helpful for a good management of the disease and for the choice of an efficient treatment. Moreover, it exists a huge amount of meaningful information in pathological reports, biological measurements and clinical information in a patient journey that remain unexploited. In that context, I propose to develop and apply novel machine learning techniques to predict cancer outcome such as recurrence or survival from multi-modal breast cancer patient data (including medical notes in natural languages and the outcome of various lab analyses). For that, I use a deep neural sequence transduction for electronic health records called BEHRT1. This model is inspired from one of the most powerful transformer-based architecture in Natural Language Processing: BERT2.
During the joint meetup with Duchess France and PyLadies Paris, Deborah Boyenval, PhD Student at Université Côté d'Azur presented a part of her PhD work: “Formal modeling of biological cyclic behavior with control points: the case of the cell cycle”.
The main limitation of biologists rooted in an experimental practice is the ability to perform rigorous proofs in the absence of a language for formalizing the biological knowledge extracted from their experiments.
Biologists have identified numerous biochemical and genetic mechanisms involved in physiological functions or diseases, but once this knowledge is linked together it remains extremely difficult and expensive to predict the impact of genetic mechanisms on physiological functions.
Déborah will present to us her thesis, which focuses firstly on a reasonable mathematical specification of complex biological functions such as cell cycle checkpoints, which represent the main barrier against cancer. Secondly, she focused on the development of an automated proof method, using the mentioned tools, proving whether a set of genetic regulations is sufficient to generate cell cycle checkpoints.
A look behind L’Oréal’s tool for consumer feedback analysis. We will discuss how NLP and Computer Vision have been applied to analyse large volumes of product reviews. Specifically, we’ll talk about topic extraction, sentiment analysis and topic enrichment in NLP, and siamese neural networks with triplet loss in Computer Vision.
"Blood flow simulation for clinical applications" by Dr Irene Vignon-Clementel, Directrice de recherche @Inria
Abstract : The dynamics of how blood flows into our body can be numerically simulated. Such simulations provide an 'augmented intelligence' to better understand cardiovascular and organ disease and plan their treatment.
More from Paris Women in Machine Learning and Data Science (20)
Sachpazis:Terzaghi Bearing Capacity Estimation in simple terms with Calculati...Dr.Costas Sachpazis
Terzaghi's soil bearing capacity theory, developed by Karl Terzaghi, is a fundamental principle in geotechnical engineering used to determine the bearing capacity of shallow foundations. This theory provides a method to calculate the ultimate bearing capacity of soil, which is the maximum load per unit area that the soil can support without undergoing shear failure. The Calculation HTML Code included.
Cosmetic shop management system project report.pdfKamal Acharya
Buying new cosmetic products is difficult. It can even be scary for those who have sensitive skin and are prone to skin trouble. The information needed to alleviate this problem is on the back of each product, but it's thought to interpret those ingredient lists unless you have a background in chemistry.
Instead of buying and hoping for the best, we can use data science to help us predict which products may be good fits for us. It includes various function programs to do the above mentioned tasks.
Data file handling has been effectively used in the program.
The automated cosmetic shop management system should deal with the automation of general workflow and administration process of the shop. The main processes of the system focus on customer's request where the system is able to search the most appropriate products and deliver it to the customers. It should help the employees to quickly identify the list of cosmetic product that have reached the minimum quantity and also keep a track of expired date for each cosmetic product. It should help the employees to find the rack number in which the product is placed.It is also Faster and more efficient way.
Overview of the fundamental roles in Hydropower generation and the components involved in wider Electrical Engineering.
This paper presents the design and construction of hydroelectric dams from the hydrologist’s survey of the valley before construction, all aspects and involved disciplines, fluid dynamics, structural engineering, generation and mains frequency regulation to the very transmission of power through the network in the United Kingdom.
Author: Robbie Edward Sayers
Collaborators and co editors: Charlie Sims and Connor Healey.
(C) 2024 Robbie E. Sayers
We have compiled the most important slides from each speaker's presentation. This year’s compilation, available for free, captures the key insights and contributions shared during the DfMAy 2024 conference.
Immunizing Image Classifiers Against Localized Adversary Attacksgerogepatton
This paper addresses the vulnerability of deep learning models, particularly convolutional neural networks
(CNN)s, to adversarial attacks and presents a proactive training technique designed to counter them. We
introduce a novel volumization algorithm, which transforms 2D images into 3D volumetric representations.
When combined with 3D convolution and deep curriculum learning optimization (CLO), itsignificantly improves
the immunity of models against localized universal attacks by up to 40%. We evaluate our proposed approach
using contemporary CNN architectures and the modified Canadian Institute for Advanced Research (CIFAR-10
and CIFAR-100) and ImageNet Large Scale Visual Recognition Challenge (ILSVRC12) datasets, showcasing
accuracy improvements over previous techniques. The results indicate that the combination of the volumetric
input and curriculum learning holds significant promise for mitigating adversarial attacks without necessitating
adversary training.
Forklift Classes Overview by Intella PartsIntella Parts
Discover the different forklift classes and their specific applications. Learn how to choose the right forklift for your needs to ensure safety, efficiency, and compliance in your operations.
For more technical information, visit our website https://intellaparts.com
Student information management system project report ii.pdfKamal Acharya
Our project explains about the student management. This project mainly explains the various actions related to student details. This project shows some ease in adding, editing and deleting the student details. It also provides a less time consuming process for viewing, adding, editing and deleting the marks of the students.
NUMERICAL SIMULATIONS OF HEAT AND MASS TRANSFER IN CONDENSING HEAT EXCHANGERS...ssuser7dcef0
Power plants release a large amount of water vapor into the
atmosphere through the stack. The flue gas can be a potential
source for obtaining much needed cooling water for a power
plant. If a power plant could recover and reuse a portion of this
moisture, it could reduce its total cooling water intake
requirement. One of the most practical way to recover water
from flue gas is to use a condensing heat exchanger. The power
plant could also recover latent heat due to condensation as well
as sensible heat due to lowering the flue gas exit temperature.
Additionally, harmful acids released from the stack can be
reduced in a condensing heat exchanger by acid condensation. reduced in a condensing heat exchanger by acid condensation.
Condensation of vapors in flue gas is a complicated
phenomenon since heat and mass transfer of water vapor and
various acids simultaneously occur in the presence of noncondensable
gases such as nitrogen and oxygen. Design of a
condenser depends on the knowledge and understanding of the
heat and mass transfer processes. A computer program for
numerical simulations of water (H2O) and sulfuric acid (H2SO4)
condensation in a flue gas condensing heat exchanger was
developed using MATLAB. Governing equations based on
mass and energy balances for the system were derived to
predict variables such as flue gas exit temperature, cooling
water outlet temperature, mole fraction and condensation rates
of water and sulfuric acid vapors. The equations were solved
using an iterative solution technique with calculations of heat
and mass transfer coefficients and physical properties.
NUMERICAL SIMULATIONS OF HEAT AND MASS TRANSFER IN CONDENSING HEAT EXCHANGERS...
Emotion Recognition in Artificial Intelligence by Valentina Franzoni, Ph.D. in Engineering for Computer Science, Research Fellow in Artificial Intelligence, adjunct professor in Operative Systems
2. - Emotions and Artificial Intelligence
- Machine Learning and Data Science
approaches
- Errors, bias, and overconfidence
in artificial emotional models
- Open Challenges
WiMLDS online meeting - June 2020 Valentina Franzoni – Emotion Recognition in Artificial Intelligence
Seminar overview
3. Emotions and Artificial Intelligence
WiMLDS online meeting - June 2020 Valentina Franzoni – Emotion Recognition in Artificial Intelligence
4. Research questions
What are emotions?
How can they be measured in
the brain?
How can they be measured in
their expressions?
How can they be represented
in computer science?
Can they be recognized using
Artificial / Web Intelligence?
WiMLDS online meeting - June 2020 Valentina Franzoni – Emotion Recognition in Artificial Intelligence
5. Cognitive system input
+
Emotional (limbic) system input
=
Decision output
What is an «emotion»
Cognitive system
Emotional system
Cognitive system
Emotional system
situations
WiMLDS online meeting - June 2020 Valentina Franzoni – Emotion Recognition in Artificial Intelligence
6. If there is a lion…
Cognitive system: “baby lion, maybe mother around.”
Emotional system: “AAAWW! CUTE!! I would like to touch it!”
Decision system: “better to watch from far.”
WiMLDS online meeting - June 2020 Valentina Franzoni – Emotion Recognition in Artificial Intelligence
7. And what if…
Cognitive system: “lions running at me!”
Emotional system: “FEAR of death! RUN AWAY!”
Decision system: “EMERGENCY!”
WiMLDS online meeting - June 2020 Valentina Franzoni – Emotion Recognition in Artificial Intelligence
8. Emotions are something that get us to
act in a particular way.
The “qualia” (feeling) is not needed to
run, but it is needed to teach and plan
for the future. It’s the internal subjective
component of sense of perception,
arising from stimulation of the senses by
phenomena
What are emotions
WiMLDS online meeting - June 2020 Valentina Franzoni – Emotion Recognition in Artificial Intelligence
9. - Primitives (modular)
- Fast (approximate)
- Executed automatically (reactive)
- Based on environment (adaptive)
- Temporary with learning (evolutionary)
- Can be collective and contagious
- NOT usual feelings or thoughts
- NOT sentiments
- NOT moods
Emotions in AI perspective
WiMLDS online meeting - June 2020 Valentina Franzoni – Emotion Recognition in Artificial Intelligence
10. Machine Learning and Data
Science Approaches
WiMLDS online meeting - June 2020 Valentina Franzoni – Emotion Recognition in Artificial Intelligence
11. 21 June-27 July 2017, Perugia
Emotions in Artificial Intelligence
Seminar series on Cyber-emotions
at the 6th Summer School
organized within Hong Kong
Baptist University and University of
Perugia, Italy
Organized by
Valentina Franzoni and Alfredo Milani
WiMLDS online meeting - June 2020 Valentina Franzoni – Emotion Recognition in Artificial Intelligence
12. WiMLDS online meeting - June 2020 Valentina Franzoni – Emotion Recognition in Artificial Intelligence
13. WiMLDS online meeting - June 2020 Valentina Franzoni – Emotion Recognition in Artificial Intelligence
14. WiMLDS online meeting - June 2020 Valentina Franzoni – Emotion Recognition in Artificial Intelligence
15. WiMLDS online meeting - June 2020 Valentina Franzoni – Emotion Recognition in Artificial Intelligence
16. WiMLDS online meeting - June 2020 Valentina Franzoni – Emotion Recognition in Artificial Intelligence
17. WiMLDS online meeting - June 2020 Valentina Franzoni – Emotion Recognition in Artificial Intelligence
18. WiMLDS online meeting - June 2020 Valentina Franzoni – Emotion Recognition in Artificial Intelligence
19. WiMLDS online meeting - June 2020 Valentina Franzoni – Emotion Recognition in Artificial Intelligence
20. Web-based proximity models
General idea:
Using search engines as a source of information about the
frequency of terms (such as Google page counts)
Observation:
• approximate the probability P(w1)=f(w1)/N where
f(w1) is the frequency, N =total of indexed objects
Ex. P(gatto)=0.000679
Advantage:
• Applicable to all proximity measures based on
probability or frequency
=cat
WiMLDS online meeting - June 2020 Valentina Franzoni – Emotion Recognition in Artificial Intelligence
21. Web-based proximity measures
to terms similarity P(x)=
f(x)
𝐍
and P(xy)=
f(xy)
𝐍
Based on probability P(x) of terms. Easy to compute by the
frequency of terms:
f(x), f(y) and f(xy) are the frequencies returned from a search
engine (e.g. Bing, Google, Yahoo, Wikipedia, Youtube) and N is
the total number of indexed documents.
WiMLDS online meeting - June 2020 Valentina Franzoni – Emotion Recognition in Artificial Intelligence
22. Vector-space Emotion Recognition
Let E be a model of emotions EEkman={anger, disgust, fear,
joy, sadness, surprise}, then the emotional state of a term
t=“kill” is 6-dimensional vector of the proximity of t from
the basic emotions.
anger
sadness
disgust
joy
fear
surprise
kill
WiMLDS online meeting - June 2020 Valentina Franzoni – Emotion Recognition in Artificial Intelligence
23. Set Proximity Emotion Recognition
Goal: to assess the emotional load of a sentence S by
evaluating the proximity of the set of terms to a set of
basic emotions.
• S={t1, t2, …,tn} terms in the target sentence
• E ={e1, e2, …,em} emotional model
v_S= (all i) f[( ti , e1), ( ti , e2), …, ( ti , em)]
the location of S in vector space model is a
functionof proximities from the reference
emotions of all theelements of the sentence.
• web based proximity measure
v_emax = max{SEL j1,n {(t1, e1),…, (tn, e1)},...,SEL j1,n {(t1, em),…,(tn, em)}}
WiMLDS online meeting - June 2020 Valentina Franzoni – Emotion Recognition in Artificial Intelligence
24. Set Proximity Emotion Recognition
WiMLDS online meeting - June 2020 Valentina Franzoni – Emotion Recognition in Artificial Intelligence
25. Emotive Face Recognition with CNN
Ekman basic Emotion
Fine-tune a pre-trained neural network (e.g., AlexNet, GoogleNet)
with a set of images related to the table of emotions (transfer
learning) and save the network
Automatically detect the position of the person's face in the camera
(e.g., via OpenCV)
Classify the emotions
Valid also for dogs
WiMLDS online meeting - June 2020 Valentina Franzoni – Emotion Recognition in Artificial Intelligence
26. Emotive Face Recognition with CNN
➢ The system is reliable and
useful for the users
➢ CNN are an excellent tool for
processing images in a fast
and accurate way
➢ Even using a commodity
webcam results are very good
➢ The system is made by open
source, well documented and
maintained tools
WiMLDS online meeting - June 2020 Valentina Franzoni – Emotion Recognition in Artificial Intelligence
27. Emotions Modeling:
The Emotographic Iceberg
Facial Expression
Context
Awareness
Prior
Experience
Words Used Body Language Voice Characteristics
Senses
Understand
Emotion
WiMLDS online meeting - June 2020 Valentina Franzoni – Emotion Recognition in Artificial Intelligence
28. WiMLDS online meeting - June 2020 Valentina Franzoni – Emotion Recognition in Artificial Intelligence
29. Computer-generated charisma:
Steve Jobs can take you anywhere!
Text-to-speech
synthesis: Synthetic
robot voice
combined with the
acoustic voice
profiles of Steve
Jobs
& Mark Zuckerberg
WiMLDS online meeting - June 2020 Valentina Franzoni – Emotion Recognition in Artificial Intelligence
30. Computer-generated charisma:
Steve Jobs can take you anywhere!
30 drivers (University students) knowing the path very well, convinced
or not to take a wrong way by the voices of Steve Jobs and Mark
Zuckerberg.
WiMLDS online meeting - June 2020 Valentina Franzoni – Emotion Recognition in Artificial Intelligence
31. Computer-generated charisma:
Steve Jobs can take you anywhere!
When the navigation system uses SJ‘s tone of voice…
26.7 % of all drivers do not abort the test drive at all but follow the
wrong instructions to the very end!
Drivers show greater trust in the system (= no error) or they do not
blame the system but others!
Perception experiments with
prosodically manipulated TTS
systems are among the first
that clearly show the power
of charismatic speech
melody in perception = not
just on listener ratings but on
listener behavior!
WiMLDS online meeting - June 2020 Valentina Franzoni – Emotion Recognition in Artificial Intelligence
32. Errors, bias, and overconfidence
in artificial emotional models
WiMLDS online meeting - June 2020 Valentina Franzoni – Emotion Recognition in Artificial Intelligence
33. Confusion of Emotion Recognition
with Mood or Sentiment Analysis
WiMLDS online meeting - June 2020 Valentina Franzoni – Emotion Recognition in Artificial Intelligence
34. • Emotions are not only feelings
• Emotions are not thoughts
• Emotions are not moods
• Emotions are transient: they come
immediately from a stimulus, reach
a peak and then gradually
degrade
Confusion of Emotion Recognition with
Mood or Sentiment Analysis
WiMLDS online meeting - June 2020 Valentina Franzoni – Emotion Recognition in Artificial Intelligence
36. Overconfidence in assuming the
emotional nature of “emotional”
machines
Naming «fear» an
if-then instruction
is functionally
imprecise…
WiMLDS online meeting - June 2020 Valentina Franzoni – Emotion Recognition in Artificial Intelligence
37. Underestimation of the effects of
well-designed emotional machines
Example: prosodic speech
WiMLDS online meeting - June 2020 Valentina Franzoni – Emotion Recognition in Artificial Intelligence
38. Underestimation of the effects of
well-designed emotional machines
WiMLDS online meeting - June 2020 Valentina Franzoni – Emotion Recognition in Artificial Intelligence
39. Duplication of concepts reinventing
the wheel
WiMLDS online meeting - June 2020 Valentina Franzoni – Emotion Recognition in Artificial Intelligence
40. Oversimplification of
human emotions
Human emotions can list
more than 130 variations.
+surprise=Ekman model
Plutchick model
WiMLDS online meeting - June 2020 Valentina Franzoni – Emotion Recognition in Artificial Intelligence
45. Emotion Recognition for Self-Aid
Possible future directions of multidisciplinary problem-
solving opportunities for in Affective Computing and
Emotion Recognition, as they emerge from the
experience of the ACER-EMORE Workshops Series.
RESEARCH PROBLEM
Enhance user consciousness of emotional states,
ultimately support the development of self-aid
applications.
WiMLDS online meeting - June 2020 Valentina Franzoni – Emotion Recognition in Artificial Intelligence
46. Problems and solutions
▪ Relevant application domains:
dependencies treatment (DT) (e.g., workaholism,
sexaholism)
non-violent communication (NVC) for people in leading
roles using e-mail or chat communication
empathy learning for parents and teachers in the circle-
of-security (COS) caring environment
▪ Suggested solutions:
facial ER using Convolutional Neural Networks
semantic text ER
▪ General framework for Emotion Recognition (ER) in
Self-aid in a human-centered approach.
WiMLDS online meeting - June 2020 Valentina Franzoni – Emotion Recognition in Artificial Intelligence
47. Why Self-Aid applications?
Ethical goals. Emotion Recognition deeply involves the
user’s intimate believes and states, sometimes even
unknown to the same user who can be unaware of
his/her emotional state.
Sound and Incremental approach. The risk is to create a
lot of models and algorithms that reach the same goal
starting from the same primary resources, instead of
advancing the already existent methods and algorithms.
Practical and Helping applications. Find practical
applications to help human beings to live better:
▪ Shortage of funds
▪ Aids for the weakest
▪ Diminish pain
WiMLDS online meeting - June 2020 Valentina Franzoni – Emotion Recognition in Artificial Intelligence
48. General Algorithm
classifying emotions from sensors data
ranking the relevance of the detected
emotions concerning the application goal and
the current context
projecting the consequence of emotions
concerning a specific AC model
eventually triggering a user emotional
feedback in order to call for a self-aid action, if
the detected emotion appears to affect the
application goals
WiMLDS online meeting - June 2020 Valentina Franzoni – Emotion Recognition in Artificial Intelligence
50. To start mastering Emotion Recognition, or for
collaboration proposals:
Valentina Franzoni
http://orcid.org/0000-0002-2972-7188
valentina.franzoni@dmi.unipg.it
http://dblp.uni-trier.de/pers/hd/f/Franzoni:Valentina
https://www.researchgate.net/profile/Valentina_Franzoni
Subscribe to (like) the Facebook page of our group:
https://www.facebook.com/affectivecomputing
WiMLDS online meeting - June 2020 Valentina Franzoni – Emotion Recognition in Artificial Intelligence
51. Appendix:
Achievement of our workshops
WiMLDS online meeting - June 2020 Valentina Franzoni – Emotion Recognition in Artificial Intelligence
52. 2 June 2017, Trieste
Emotions in Artificial Intelligence
International Multidisciplinary
workshop on Emotion Recognition
(EMORE) at the 17th International
Conference on Computational
Science and Its Applications
(ICCSA 2017)
Organized by:
Valentina Franzoni, Daniele Nardi, Alfredo Milani, Jordi Vallverdú
WiMLDS online meeting - June 2020 Valentina Franzoni – Emotion Recognition in Artificial Intelligence
53. Giulio Biondi, Valentina Franzoni, Valentina Poggioni:
A Deep Learning Semantic Approach to Emotion Recognition Using the IBM
Watson Bluemix Alchemy Language. ICCSA (3) 2017: 718-729
Valentina Franzoni, Giulio Biondi, Alfredo Milani:
A Web-Based System for Emotion Vector Extraction. ICCSA (3) 2017: 653-668
Matteo Riganelli, Valentina Franzoni, Osvaldo Gervasi, Sergio Tasso:
EmEx, a Tool for Automated Emotive Face Recognition Using Convolutional
Neural Networks. ICCSA (3) 2017: 692-704
Banu Yergesh, Gulmira Bekmanova, Altynbek Sharipbay, Manas Yergesh:
Ontology-Based Sentiment Analysis of Kazakh Sentences. ICCSA (3) 2017:669-
677
Zhiyi Ma, Marwa Mahmoud, Peter Robinson, Eduardo Dias, Lee Skrypchuk:
Automatic Detection of a Driver’s Complex Mental States. ICCSA (3) 2017:678-
691
Francesca D’Errico, Isabella Poggi:
“Humble” Politicians and Their Multimodal Communication. ICCSA (3) 2017:705-
717
Published in In Proceedings of ICCSA ’17, Trieste, Italy, July 3-6, 2017. LNCS.
WiMLDS online meeting - June 2020 Valentina Franzoni – Emotion Recognition in Artificial Intelligence
54. 14 July 2017, Perugia University
Emotions in Artificial Intelligence
Valentina Franzoni, chair
Giulio Biondi, co-chair & techs
Jordi Vallverdú, Spain
Fabio Massimo Botti, Italy
Radoslaw Niewiadovmski, Poland
Alfredo Milani, Italy
Lalit Garg, Malta
Valentina Poggioni, Italy
Workgroup on Affective
Computing (remote kick-off)
WiMLDS online meeting - June 2020 Valentina Franzoni – Emotion Recognition in Artificial Intelligence
55. 23 August 2017, Leipzig
Emotions in Artificial Intelligence
International Workshop on
Affective Computing and Emotion
Recognition (ACER) at the
IEEE/WIC/ACM International
Conference on Web Intelligence
(WI 2017)
Organized by:
Valentina Franzoni, Jiming Liu, Alfredo Milani, Daniele Nardi, Roberto Basili
WiMLDS online meeting - June 2020 Valentina Franzoni – Emotion Recognition in Artificial Intelligence
56. Published in In Proceedings of WI ’17, Leipzig, Germany, August 23-26, 2017.
Valentina Franzoni, Alfredo Milani, Giulio Biondi:
SEMO: a semantic model for emotion recognition in web objects. WI 2017: 953-
958. DOI: 10.1145/3106426.3109417
Valentina Franzoni, Valentina Poggioni:
Emotional book classification from book blurbs. WI 2017: 931-938.
DOI: 10.1145/3106426.3109422
Juan Manuel Mayor Torres, Evgeny A. Stepanov: Enhanced face/audio emotion
recognition: video and instance level classification using ConvNets and
restricted Boltzmann Machines. 939-946. DOI: 10.1145/3106426.3109423
Valentina Franzoni, Alfredo Milani, Jordi Vallverdú:
Emotional affordances in human-machine interactive planning and
negotiation. WI 2017: 924-930 DOI: 10.1145/3106426.3109421
Alejandro Hernández-García, F. Fernández-Martínez, Fernando Díaz-de-María:
Emotion and attention: predicting electrodermal activity through video visual
descriptors. 914-923. DOI: 10.1145/3106426.3109418
Valentina Franzoni, Yuanxi Li, Paolo Mengoni: A path-based model for emotion
abstraction on facebook using sentiment analysis and taxonomy knowledge. WI
2017: 947-952. DOI: 10.1145/3106426.3109420
WiMLDS online meeting - June 2020 Valentina Franzoni – Emotion Recognition in Artificial Intelligence
57. January 2018
Special issue on
Emotional Machines:
the Next Revolution
in the International Journal on
Web Intelligence (iOS press)
Guest editors:
Valentina Franzoni, Alfredo Milani, Daniele Nardi, Jordi Vallverdù
WiMLDS online meeting - June 2020 Valentina Franzoni – Emotion Recognition in Artificial Intelligence
58. Published in «Emotional Machines: the Next Revolution», Web Intelligence 2018
https://content.iospress.com/journals/web-intelligence/17/1
Valentina Franzoni, Alfredo Milani, Daniele Nardi, Jordi Vallverdú
Emotional machines: The next revolution (editorial)
https://content.iospress.com/art…/web-intelligence/web190395
Banu Yergesh, Gulmira Bekmanova, Altynbek Sharipbay
Sentiment analysis of Kazakh text and their polarity
Osvaldo Gervasi, Valentina Franzoni, Matteo riganelli, Sergio Tasso
Automating facial emotion recognition
Álvaro García-Faura, Alejandro Hernández-García, et al.
Emotion and attention: Audiovisual models for group-level skin response
recognition in short movies
Karan Sharma, Marius Wagner, Claudio Castellini, Egon L. Van den Broek et al.
A functional data analysis approach for continuous 2-D emotion annotations
Alessandro Ansani, Francesca D’Errico, Isabella Poggi
‘You will be judged by the music I hear’: A study on the influence of music on
moral judgement
Francesca D’Errico, Isabella Poggi
Tracking a leader’s humility and its emotions from body, face and voice
Andrea Gorrini, Luca Crociani, Giuseppe Vizzari, Stefania Baldini
Stress estimation in pedestrian crowds: Experimental data and simulations results
WiMLDS online meeting - June 2020 Valentina Franzoni – Emotion Recognition in Artificial Intelligence
59. 3 July 2018, Sidney, Australia
International Multidisciplinary
workshop on Emotion Recognition
(EMORE) at the 18th International
Conference on Computational
Science and Its Applications
(ICCSA 2018)
Organized by:
Valentina Franzoni, Alfredo Milani
Co-located with WCES
WiMLDS online meeting - June 2020 Valentina Franzoni – Emotion Recognition in Artificial Intelligence
60. 3 July 2019, Saint Petersburg,
Russia
International Multidisciplinary
workshop on Emotion Recognition
and Affective Computing
(ACER/EMORE) at the 19th
International Conference on
Computational Science and Its
Applications (ICCSA 2019)
Organized by:
Valentina Franzoni, Alfredo Milani, Giulio Biondi
WiMLDS online meeting - June 2020 Valentina Franzoni – Emotion Recognition in Artificial Intelligence
61. Valentina Franzoni, Alfredo Milani:
Emotion Recognition for Self-aid in Addiction Treatment, Psychotherapy, and
Nonviolent Communication. ICCSA (2) 2019: 391-404
Giulio Biondi, Valentina Franzoni, Osvaldo Gervasi, Damiano Perri:
An Approach for Improving Automatic Mouth Emotion Recognition. ICCSA (1)
2019: 649-664
Oliver Niebuhr, Jan Michalsky:
Computer-Generated Speaker Charisma and Its Effects on Human Actions in a
Car-Navigation System Experiment - or How Steve Jobs' Tone of Voice Can
Take You Anywhere. ICCSA (2) 2019: 375-390
Francesca D'Errico, Oliver Niebuhr, Isabella Poggi:
Humble Voices in Political Communication: A Speech Analysis Across Two
Cultures. ICCSA (2) 2019: 361-374
Chutisant Kerdvibulvech, Sheng-Uei Guan:
Affective Computing for Enhancing Affective Touch-Based Communication
Through Extended Reality. ICCSA (2) 2019: 351-360
Eoghan Furey, Juanita Blue:
The Emotographic Iceberg: Modelling Deep Emotional Affects Intelligent
Assistants & the IoT. ICCSA (7) 2019: 175-180
Published in In Proceedings of ICCSA’19, S.Petersburg Russia, July 3-6, 2019. LNCS.
WiMLDS online meeting - June 2020 Valentina Franzoni – Emotion Recognition in Artificial Intelligence
62. 14 Oct 2019, Thessaloniki,
Greece
International workshop on
Affective Computing and
Emotion Recognition (ACER)
at ACM/IEEE/WIC Web
Intelligence Conference
(WI2019)
Organized by:
Valentina Franzoni, Alfredo Milani, Daniele Nardi, Jordi Vallverdù
WiMLDS online meeting - June 2020 Valentina Franzoni – Emotion Recognition in Artificial Intelligence
63. Valentina Franzoni, Jordi Vallverdú, Alfredo Milani:
Errors, Biases and Overconfidence in Artificial Emotional Modeling.
WI (Companion) 2019: 86-90
Valentina Franzoni, Alfredo Milani, Giulio Biondi, Francesco Micheli:
A Preliminary Work on Dog Emotion Recognition.
WI (Companion) 2019: 91-96
Sergio Angelastro, Berardina De Carolis, Stefano Ferilli:
Learning and Predicting User Pairwise Preferences from Emotions and Gaze
Behavior.
WI (Companion) 2019: 72-79
Berardina De Carolis, Francesca D'Errico, Nicola Macchiarulo, Giuseppe
Palestra:
"Engaged Faces": Measuring and Monitoring Student Engagement from Face
and Gaze Behavior.
WI (Companion) 2019: 80-85
Published in In Proceedings of WI ’19, Thessaloniki, Greece, October 14, 2019.
WiMLDS online meeting - June 2020 Valentina Franzoni – Emotion Recognition in Artificial Intelligence
64. 6 Oct 2019, Bari, Italy
International Workshop on Socio-
Affective Technologies: an
interdisciplinary approach (SAT)
at IEEE Systems, Man, and
Cybernetics Conference
(SMC 2019)
Organized by:
Berardina De Carolis, Francesca D’Errico, Veronica Rossano
WiMLDS online meeting - June 2020 Valentina Franzoni – Emotion Recognition in Artificial Intelligence
65. Valentina Franzoni, Giulio Biondi, Alfredo Milani: Crowd emotional sounds: spectrogram-
based analysis using convolutional neural network. SAT@SMC 2019: 32-36
Concetta Papapicco, Giuseppe Mininni: Impact memes: PhDs humor(e). SAT@SMC 2019:
1-6
Giovanni Boccia Artieri, Gevisa La Rocca: The election day of Pope Francis: between
sentiment and emotions online. SAT@SMC 2019: 7-12
Marinella Paciello, Francesca D'Errico, Giorgia Saleri: Moral struggles in social media
discussion: the case of sexist aggression. SAT@SMC 2019: 13-16
Francesca D'Errico, Manuel Martinez, Carmen D'Anna, Maurizio Schmid, Stella
Mastrobattista, Raffaella Parlongo, Christian Massom: 'Prosocial' virtual reality as tool for
monitoring engagement in intergroup helping situations. SAT@SMC 2019: 17-21
Fabrizio Balducci, Berardina De Carolis, Donato Impedovo, Giuseppe Pirlo: Touch
dynamics for affective states recognition: your smartphone knows how you feel since you
unlock it. SAT@SMC 2019: 22-26
Anne-Marie Brouwer, Ivo V. Stuldreher, Nattapong Thammasan: Shared attention
reflected in EEG, electrodermal activity and heart rate. SAT@SMC 2019: 27-31
Mauro Gaspari, Margherita Donnici: Weekend in Rome: a cognitive training exercise
based on planning. SAT@SMC 2019: 37-41
Berardina De Carolis, Giuseppe Palestra, Olimpia Pino: Facial expression recognition from
nao robot within a memory training program for individuals with mild cognitive
impairment. SAT@SMC 2019: 42-46
Published in In Proceedings of SAT ’19, Bari, Italy, October 6, 2019.
WiMLDS online meeting - June 2020 Valentina Franzoni – Emotion Recognition in Artificial Intelligence
66. 3 July 2020, Cagliari, Italy (online
for Covid-19)
International Multidisciplinary
workshop on Emotion Recognition
and Affective Computing
(ACER/EMORE) at the 20th
International Conference on
Computational Science and Its
Applications (ICCSA 2020)
Organized by:
Valentina Franzoni, Alfredo Milani, Giulio Biondi
WiMLDS online meeting - June 2020 Valentina Franzoni – Emotion Recognition in Artificial Intelligence
67. These slides are distributed
under the Creative Commons
licence:
CC BY-SA 4.0
You can use, share and modify
them under the constraints to
cite the original author and
share with the same licence:
WiMLDS online meeting - June 2020 Valentina Franzoni – Emotion Recognition in Artificial Intelligence
https://creativecommons.org
/licenses/by-sa/4.0/deed.en