Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Quantum Lab White Paper

1,627 views

Published on

Emotion recognition technology White Paper written by Quantum Lab. Describes uses of Emotion Recognition technology in fields of advertisement testing and Customer Experience. Describes methods of validation.

Published in: Data & Analytics
  • Be the first to comment

Quantum Lab White Paper

  1. 1. WHITE PAPERby QUANTUM LAB
  2. 2. We believe that most of the problems in the world, business and personal, are the result of human nature. We believe that in order to change something, you have to be able to measure it. We believe that in order to measure something, you have to understand it. We believe that emotions inspire behaviors and inaction.
  3. 3. Therefore, our mission is to enable businesses to automatically understand human nature and make a positive change where it was not previously possible. Bartosz Rychlicki, CEO and Co-Founder Quantum Lab
  4. 4. Contents 4 Contents 1. Quantum Lab    5 1.1. Mission and vision    6 1.2. Quantum Lab experts    7 2. Quantum Sense    9 2.1. Technology    10 2.2. Measurement effectiveness accreditation    13 2.3. Quantum Sense vs. Human    15 3. Quantum Insight    17 3.1. The power of emotional advertisement    18 3.2. Quantum Insight    22 3.3. Insight Score    23 3.4. Quantum Insight Applications    24 3.5. Presentation of results    28 3.6. Quantum Insight Functionality    30 4. The final word    31 5. Awards    32 6. Bibliography    33
  5. 5. Quantum Lab 01
  6. 6. Quantum Lab 6 MISSION AND VISION The issue of emotions was the subject of discussion for many prominent thinkers and scientists over the centu- ries. The golden era of research in emotions fell in the se- cond half of the last century, along with the work of the pioneer in the field of mobile-mimic expression of emo- tions - Paul Ekman. The same period witnessed the deve- lopment of the explorations of the phenomenon of adver- tising. The effects of the work of that time gave birth to the phrase „good publicity is emotional publicity.” In the year 2013 the fascination and passion for those subjects resulted in forming the Quantum Lab Company. Company directors and managers reported a need for support in issues related to human behavior. Thus our mission was established. We recognize the problems of business orga- nizations and provide them with an effective solution thro- ugh automation, psychological know-how and technology. We wanted to apply psychological knowledge in practi- ce in order to allow simple use of its resources. We know the rules allowing for the analysis of emotions and beha- vioral responses. We were able to teach this to computers. We track and measure emotions on the basis of scientific knowledge from the field of affective computer science and psychology. The continuous increase in computing po- wer is opening up new opportunities for us to implement more complex algorithms and processes. We don’t just generate data, but also make them understandable and operable. The basis of this process is the Quantum Sense technology, designed by us. It enables the analysis of emotions and behavior of people in real time. Based on this, we have built two tools - Quantum Insight and Quantum CX which, by means of a camera, recognize and classify emotional and behavioral reactions, presen- ting them in the form of objective data. The first of these solutions is used primarily in marketing, while Quantum CX works in the retail area. The foreign branch of the com- pany is located in the USA. Our solutions are used by va- rious research institutions and academic circles. From day to day our technology becomes more and more „human”, in the context of understanding of what it „sees.” In the near future we want to make it able to learn automatically. In this way, we shall implement a kind of experience in it. We want to be able to obtain information from multiple channels (e.g. sound), thereby acquiring other „senses”. Multidimensional, aggregated data would com- plement the technology, greatly enhancing its capabilities.
  7. 7. Quantum Lab 7 QUANTUM LAB EXPERTS The cardinal advantage of our technology and tools built based on it is our multidisciplinary and talented team of R & D Quantum Lab specialists. KRZYSZTOF CYWIŃSKI Chief Technology Officer - directs production, is respon- sible for the development of applications and analyzing machine data. Deals with the design of neural networks that recognize and analyze emotions and behavioral in- dicators, and their implementation in sectors based on non-verbal communication, emotions and human behavior. Krzysztof is the most rational part of the team and there is nothing impossible for him to do. DARIUSZ KAMOWSKI Our AI Software Engineer, supports the entire process, dealing with the analysis of the received data and process optimization and diagnostics, based on the use of artifi- cial intelligence. He is an enthusiast, a winner and finalist of competitions in the areas of electrical power, automa- tion, nanotechnology and information technology. He is responsible for implementation of solutions in the field of computational intelligence and digital signal processing.
  8. 8. Quantum Lab 8 JUSTYNA LESZCZ Research Specialist, psychologist. Author of projects in the area of non-verbal communication and emotional intelligence. Her passion is people and statistics. In Quantum Lab, she takes care of the scientific basis for projects and is respon-sible for the development and validation of solutions to the problems of human nature with psychological methods. Thanks to close cooperation in the development and im- plementation of projects, they contribute comprehensive knowledge and experience in the area of artificial intelli- gence, psychology and affective computing to the tech- nologies and products of Quantum Lab, thereby granting them a prominent place in the world of methods for auto- matic analysis of human behavior and emotions.
  9. 9. Quantum Sense 9 Quantum Sense 02
  10. 10. Quantum Sense 10 happiness disgust anger sadness neutral surprise TECHNOLOGY Quantum Sense is a  proprietary technology developed by Quantum Lab, which allows an automatic analysis of behavioral and emotional indicators occurring on a human face. By means of micro-expressions detection, it effectively recognizes five basic emotions: joy, sadness, surprise, disgust, anger and lack of emotion, called the neutral state. Using computer vision and artificial intel- ligence, our technology classifies emotions arising in re- sponse to a specific stimulus (e.g. video). Besides the analysis of facial expression, also used is the analysis of non-verbal factors, such as body positioning, gestures, posture, general mobility and ove- rall response time to certain stimuli. Our invention will soon also be able to verify verbal factors, e.g. acoustic speech parameters, patterns of speech rate, the con- tent of speech or non-linguistic sound responses (such as sighs, shouts or yawns).
  11. 11. Quantum Sense 11 need stimulusemotion reaction level of satisfaction Due to use of this modern technology, the analysis process of emotion has never been so simple. The test is non-invasive - it does not require any devices attached to the human body. To make a measurement, only a standard camera is required, available in most laptops. The analysis of facial expressions may be used alone or in combination with biometric sensors such as EEG, GSR, EMG, FMRI or using eye tracking devices. The use of Quantum Sense allows to determine the true opinion about a particular experience, based on the emotional reaction of customers and recognized patterns of behavior. The use of measure- ments instead of declarations eliminates the subconscious resistance of clients to expressing negative judgments and allows to avoid cognitive errors. The process of Quantum Sense operation can be presen- ted using the following steps: • Faces and their positions are located in the source ma- terial, which can be a video recording, a camera image or a photograph. The search process is supported by a very thorough and fast algorithm, so even a significant deviation from the frontal position is acceptable. • In the next step, the area of the face is normalized - among other things, the face is scaled, centered, and subjected to filtration, with particular recognition of lighting compen- sation and impact of movement. This treatment allows for a significant correction or complete elimination of the in- fluence of light, and the correction of the face positioning.
  12. 12. Quantum Sense 12 1. 2. 3. FINDING THE FACE In the first step, the algorithms find and define the position of the face. CLASSIFICATION OF EMOTIONS Automatic classification of facial points according to known expressions of emotion. EXTRACT CHARACTERISTIC POINTS Arranged, among others, around the eyebrows, eyes, nose, mouth. • A face thus „prepared” is subjected to an algorithm, al- lowing for separation of its characteristics and physical properties. An active shape model is the applied to the face image. This method is able to identify 68 specific points on the face. This process allows the determination of how various points behave during facial expressions, appearing under the influence of emotions. • Subsequently, the pre-prepared machine learning algori- thms, based on the use of artificial intelligence, come to our aid. Thanks to them we are able to translate the above dependencies to the automatic classification of emo- tions for the face, which were not visible in the process of learning classifiers. Quantum Sense uses a variety of computational intelligence techniques for this purpose, dominant among which are: • the support vector machine - SVM, • the artificial neural network, • various algorithms from the deep learning family, e.g. Convolutional Neural Networks - CNN. Based on Quantum Lab technologies, we have deve- loped two tools: Quantum CX (Customer Experience) and Quantum Insight. The first of these is a tool that supports managers and restaurant owners. It is used to measure the level of service quality and levels of guest satisfaction. It is able to identify these two parameters on the basis of data such as talk time, eye contact, presence in the work-
  13. 13. Quantum Sense 13 place. Thanks to our technology managers may, in an easy, fast and automated way diagnose and subsequently im- prove, along with the staff, the style of service, which wo- uld be beneficial to everyone involved. The Quantum Lab technology also has a number of applications in various fields and sectors of market and research. In focus gro- ups, it allows to detect emotional reactions to the physical product or advertising. It also guarantees objective client opinion polls’ results (based on subconscious reactions, not on questions). Our technology may also be used as a  security tool, e.g. at airports, mass events or in vehicles. It allows for detection of frustration / aggression or tired- ness of the car or bus driver. Quantum Sense is an effective tool for the analysis of frustration when using products or services. It allows real-time matching of services and digital products to the feelings of users (e.g. a game that changes its story depending on the reaction of the players). Another area in which our technology may find a variety of applica- tions is robotics. Due to copyright solutions proposed by Quantum Lab, in the future we may be able to grant robots with empathy skills. MEASUREMENT EFFECTIVENESS ACCREDITATION The process of assigning and detection of micro-expres- sion on the basis of a camera image is a complex pro- cess. To be able to claim precise obtained results, we have developed rigorous measurement and data processing steps. This methodology grants us complete confidence in the quantitative results obtained and offers the possi- bility of their qualitative analysis. QUANTUM SENSE VALIDATION Cross-validation, otherwise known as a cross-testing, allows to specify how the classifier trained model will behave in the event of data that were not used in its con- struction. In other words, we test the accuracy with which Quantum Sense determines facial emotions of a particu- lar face, based on the classifier learned on a database, created by a team of Quantum Lab specialists. To obtain a reliable result in the use of cross-valida- tion, we apply k-fold validation. The original learning set is divided into K subsets. Then, sequentially, each of them serves as a test set, and the rest as a training set. Further analysis is performed. Analysis is thus performed K times. Then K results are averaged to obtain a single result, defi- ned in this case as a result of cross-validation. Quantum Sense obtains up to 95.7% by this method, which puts it at the forefront of technologies used for automatic ana- lysis of facial expression. The cross-validation method is reliable and widely used in technical and information sciences. Aware of the importance of delivered results and their accuracy, we have decided to go a step further than others and deve- lop a creative tool to verify that the procedures used by us actually lead to the planned results.
  14. 14. Quantum Sense 14 fig. 1. Quantum CX built on Quantum Sense technology currently used in retail and security. fig. 2. Quantum Insight built on Quantum Sense is a marketing tool used by research agencies, media houses and TV stations.
  15. 15. Quantum Sense 15 THE TEST PLATFORM Along our way to adapt our technology to the highest qu- ality standards, we have agreed that the cross-validation procedure in itself is insufficient for the improvement of algorithms. From this need arose our copyright solu- tion, allowing to specify the increase in effectiveness of subsequent versions of the emotions recognition engine – the test platform. It enables the analysis of real video recordings by com- petent judges. With its use, with an accuracy to a single frame, the emotional reaction of the respondents in the camera frame is determined by experts. In this way, we obtain an objective assessment of the subjects’ emotions. Then, the results from the technology of automatic reco- gnition of emotions and the determinations made by the judges are compared. According to the golden rule guiding our actions - „if you don’t measure something, you cannot improve it,” we have obtained hard evidence of accuracy of Quantum Insight in the form of research results. The development and validation of the double me- ans of validating the results aroused our curiosity to check the accuracy in recognition of human emotions by humans themselves. To have a clear picture of the effec- tiveness of our technology and the natural human ability, we designed a study, which clearly defined the efficiency of Quantum Sense. QUANTUM SENSE VS. HUMAN Intrigued by the fact that the measurement of emotional in- volvement helps to prepare an effective marketing campaign, we strive for continuous improvement of our technology. To this end, we have conducted a study comparing the effecti- veness in detecting facial micro-expressions by our techno- logy and by the human eye. We selected two video recordings containing the emo- tional reactions of people in the frame. These reactions have been identified and marked by competent judges, using spe- cialized software. Subsequently, the respondents were pre- sented with both materials, one after the other. The task of the study subject was to determine what emotion is expres- sed by the subject on video. The researcher had to mark, in real time, second by second, the emotion chosen from the list by the participant: 1. Anger; 2. Disgust; 3. Joy; 4. Grief; 5. Surprise; 6. Neutral. The same films were analyzed by the Quantum Sense technology. THE RESULTS WERE SURPRISING The average result for all respondents in both films is 46,33% efficiency, with men showing greater accura- cy in determining emotions than women (the average for women was 44.96%, while men scored 47.96%). Quantum Sense recognize, on the other hand, recognized emotions with the efficiency of 83.28%. Thus, the test results con-
  16. 16. Quantum Sense 16 firmed an effectiveness of our technology in the diagno- sis of human emotions twice better, when compared with the skills of an average person. The long-lasting work on designing and validating Quantum Sense and its satisfactory results, finally allowed for the commercial application of technology developed by Quantum Lab. We could hardly wait to grant the possibility of solving the problems of human nature to anyone for whom it constituted a daily challenge. The first tool that our clients could use the benefits of was Quantum Insight. fig.3. The results of the effectiveness of detection of 5 basic emotions and the neutral state by humans and the Quantum Sense technology. vs. Human
  17. 17. Quantum Insight 03
  18. 18. Quantum Insight 18 THE POWER OF EMOTIONAL ADVERTISEMENT The impact of emotions on the daily decisions the average person is fascinating - they determine the purchase of a ju- ice to drink, a shampoo or a specific car model, as well as the choice of a career or a life partner. The phenomenon of emotions lies in the fact that, as one of the three atti- tude-forming components, they imply our judgments, and ultimately translate to actions (McDuff, Kaliouby, Senechal, Demirdjian & Picard, 2014). Research conducted by Bradley et al. clearly shows the fact, that the key role in remembe- ring the presented objects is played by arousal and pleasure from exposure to the material. In other words, content eli- citing an emotional response is better stored in memory and subsequently remembered more easily. „Emotional campaigns have almost twice the potential of achieving great financial effects of the rational ones – even in rational categories” Binet and Field The aforementioned principles have been used in mar- keting. As indicated by the results of Berger and Milkman, up to 85% of today’s audiences consume video for en- tertainment, relaxation or excitement. A good video is, above all, about emotions. Similarly speaking, effective advertising is based on the appropriate dosage to viewers. Measuring emotional involvement helps to prepare an ef- fective marketing campaign. In this process, not only know- ledge matters, but also the appropriate tools to determi- ne what emotions the material elicits (Teixeira, Wedel & Pieters, 2012). Practice shows that advertising that triggers very emotional responses from viewers is more effective, in comparison with materials that are neutral for the reci- pient. The research by Millward Brown - an international company engaged in market research and public opinion -covered an analysis of over 12,000 facial expressions. Expertise has shown that facial expressions are a relia- ble predictor of the degree of viewer sympathy for the ad. Moreover, on this basis, we can predict the intention to purchase the advertised product. Emotional processes play a fundamental role in the ad- vertising message, as the process of perception, memo- ry and recall depends on them. Emotional ads are more effective than rational messages – according to Binet and Field - experts in modern marketing. In „The Long and the Short of it” in Harvard Business Review, they explain how
  19. 19. Quantum Insight 19 How emotions affect the planning and effectiveness of advertising? New empirical data shows that emotional advertising model and measuring the contents emotional response, lead to greater efficiency, effectiveness, better planning and decision-making during content design. Wood, O., Using an emotional model to improve the measurement of advertising effectiveness. BrainJuicer. emotional involvement of the viewer enhances loyalty, at the same time reducing the price sensitivity of the custo- mers, and thus, bringing twice the profits. In other words, building an emotional connection with the brand makes the rational message better reach the potential customers. It is worth to refer to reflection at this stage, how much more effective advertising would be, if we were able to predict what emotions, at which time and in what combinations should appear to the recipient to create an emotional connection with the brand, and consequently buy the presented product and recommend a particular brand to their friends. QUANTUM INSIGHT ALREADY KNOWNS ALL THAT How much rationality and how much emotions? Emotional ads are much more effective than rational content, especially in the long perspective. They generate twice as much profit and can sustain it much longer than a rational message Binet, L., Field, P. (2013). The long and the short of it: 10 key principles of success. Raport Instytutu Praktyków Reklamy (IPA). FACT 1 FACT 2
  20. 20. Quantum Insight 20 THE INCREASED VALUE OF FULLY CONNECTED CUSTOMERS RELATIVE TO HIGHLY SATISFIED ONES VARIES BY CATEGORY. HERE ARE THE VALUES FOR THE NINE CATEGORIES SAMPLED. HOUSEHOLD CLEANER PURCHASES +103% TABLET APP PURCHASES +82% CREDIT CARD SWLPES +68% ONLINE RETAILER PURCHASES +52% HOTEL ROOM STAYS +41% DISCOUNT STARE VISITS +37% CONSUM- ERBANKING PRODUCTS +35% FASTFOOD VISITS +27% CASINO- GAMING SPENDING +23 CUSTOMER VALUE in relation to highly satisfied customers -18% BASELINE +13% +52% NOT EMO- TIONAUY CONNECTED HIGHLY SATISFIED but not fully connected PERCEIVE BRAND DIFFEREN- TIATION and satisfied, but not fu lly con- nected FUUY CON- NECTED and satisfied, and able to per- ceive brand differentiati on Customers who feel an emotional connection to the brand are on average 52% more valuable income-wise compared to customers that are merely satisfied. Magids, S., Zorfas, A., Leemon, D. (2015). The New Science of Customer Emotions. In: Harvard Business Review. Noweber, 2015 THE VALUE OF EMOTIONAL CONNECTION AS CUSTOMERS’ RELATIONSHIPS WITH A BRAND DEEPENS, THEY MOVE ALONG THE PATHWAY TOWARD FULL EMOTIONAL CONNECTION. ALTHOUGH THEY BECOME MORE VALUABLE AT EACH STEP, THERE’S A DRA- MATIC INCREASE AT THE FINAL ANE: ACROSS A SAMPLE OF NINE CATEGORIES, FULLY CONNECTED CUS- TOMERS ARE 52% MORE VALUABLE, ON AVERAGE, THAN THOSE WHO ARE JUST HIGHLY SATISFIED. SOURCE SCOTT MAGIDS, ALAN ZORFAS, AND DANIEL LEEMON FROM “THE NEW SCIENCE OF CUSTOMER EMOTIONS,” NOVEMBER 2015 FACT 3
  21. 21. Quantum Insight 21 Should my advertisement evoke emotional reactions? The Millward Brown study showed that the more emotions appear in the ad, the better the ad is remembered and the greater consumer involvement it generates, which in turn translates into more sales. Brown, M., (2009). Should My Advertising Stimulate an Emotional Response? Millward Brown: Knowledge Point. What makes on-line content viral? Research shows that the key to effective content virality is emotional engagement. It turns out that content that triggers high emotional engagement is more likely to be shared. Berger, J., Milkman, K. (2012). What Makes online Content Viral? Journal of Marketing Research, Vol. 49, No. 2, pp. 192-205 Emotional involvement when watching internet advertising. Studies have shown that the emotions of surprise and joy focus consumers’ attention on the ad for longer and therefore make them stay in front of a computer screen for longer. Teixeira, T., Wedel, M., and Pieters, R. (2012). Emotion-induced engagement in internet video ads. Journal of Marketing Research, Vol. 49, No. 2, pp. 144-159. The key role of emotional arousal and pleasure in making the ad stick in consumers’ memory. It is easier to remember and recall content that evokes strong emotional arousal. Bradley, M. M., Greenwald, M. K., Petry, M. C., Lang, P. J. (1992). Remembering pictures: Pleasure and arousal in memory. Journal of Experimental Psychology: Learning, Memory, & Cognition 18 (2): 379–390. The analysis of more than 12,000 facial expressions has showed that it is possible to use them in order to predict the extent to which consumers will like the ad and will be willing to buy the product it advertises. McDuff, D., El Kalioubi, R., Cohn, J. F., & Picard, R. (Accepted with revisions). Predicting ad liking and purchase intent: Large-scale analysis of facial responses to ads. IEEE Transactions on Affective Computing. FACT 4 FACT 5 FACT 6 FACT 7 FACT 8
  22. 22. Quantum Insight 22 QUANTUM INSIGHT Theexpressionpatternsofsixbasicemotionsareintercultural, universal. Quantum Insight is able to recognize five of them – anger, disgust, joy, sadness and surprise and, in addition, the neutral state. It is based on the Facial Action Coding System (FACS). This system enables the identification of facial emotional reactions, appearing in the form of micro-expressions. The mimic reaction appears automatically in response to an outside stimulus. The response time is too short to register it each time and, consequently, to effectively recognize emotions. Quantum Insight comes to aid here, free of such limitations (it uses a camera to “look” at the world), perfectly capturing every micro-expression appearing on the face of the observer. Thanks to intelligent algorithms, Quantum Insight can recognize the emotions of the audience watching an advertisement spot with an accuracy of 95.7%. Courtesy of Quantum Insight, we can measure the emotional load of a marketing message, as well as provide real-time data. Such on-line procedure has a number of advantages - the research group can be made up of respon- dents from around the world, and the results are available in as little as 72 hours from the moment of placing an order. In addition, the natural conditions for making decisions protect against the influence of disturbing factors on the reactions of respondents. The whole survey takes place in the respondent’s home In any place in the world Measures natural and spontaneous emotional reactions of viewers Quantum Insight platform runs within the cloud Data is provided in real time, and all results are available 72 hours after the test
  23. 23. Quantum Insight 23 INSIGHT SCORE The Insight Score is a proprietary algorithm developed by Quantum Lab. It defines the emotional structure of advertising - analyzes the frequency of consumers’ posi- tive emotions during the presentation of a video material. In the same way, it measures negative emotions and the periods in which consumers do not express any emotions whatsoever. This process takes into account the position of the brand in the context of an emotional reaction. Our proprietary algorithm takes into account the dynamics of emotions of viewers and their trend in the sections that have an impact on the perception of advertising. Insight Score is an advanced index, calculated according to our developed formulas. To ensure the highest quality of our services, it is constantly tested and improved. Its big advantage is that it allows for effective evaluation of advertising in terms of viewers, called the positive emotional involvement. It reacts to changes in trends in advertising and refers to the actual values of the market. The higher the score, the higher the emotional involvement of viewers while watching the ad and a more positive response to branding. With Insight Score, we may not only quickly and economically compare different versions of the same ad, but also confront our own productions with the materials produced by the competition. max 7 max 7 PUBLIC SECTOR PURCHASE 1910 FIT 5,05 5,51 4,97 5,23 WOMEN MEN max 7 max 7 11 THE MEDIAN FOR THE 29 ADS
  24. 24. Quantum Insight 24 QUANTUM INSIGHT APPLICATIONS THE SURVEY PROCESS An important factor in studies using the Quantum Insight platform is the appropriate preparation of the condi- tions in which they are held. The test is carried out in the respondent’s home, and therefore, before starting the test procedure, each of the test subjects is instructed by a manual, especially developed for this purpose. The ideal measurement conditions include: • lack of back lighting, • lack of side lighting, • face set perpendicular to the camera, • face clearly visible. The Quantum Insight system includes triple control of the measurement conditions. The observation of the face on the screen allows the respondent to evaluate the accuracy of the measurement conditions. Next, the system automatically checks whether the test subject meets the- se requirements and enables them to move to the target phase of the study, or asks for a correction according to the guidelines. During the study, the platform verifies the quality of the detected face and asks the viewer for a cor- rection of the head position or interrupts the test if there is no re-adjustment to the conditions. Subsequently, the video is sent to the respondent for the purpose of analysis. fig. 4. Evaluation and control of the measuring conditions before and during the study.
  25. 25. Quantum Insight 25 In developing the Quantum Insight platform, we set ourselves the target to minimize, as much as possible, the impact of negative conditions that may interfere with the testing process. Accordingly, the artificial intelligence algorithms are trained in effective emotion recognition on both studio images and recordings failing to meet the ideal requirements (among other things - in a room that is badly lit, with subjects that are bearded, tattooed, wearing glasses etc.). Quantum Lab has an extensive database of facial images. Using Quantum Sense, so far more than 7.5 million video frames were analyzed, with more than 80,000 unique faces. We are constantly enlarging our resources. fig.5. Using specially trained algorithms, Quantum Insight recognized emotions also in tough test conditions, such as during the consumption of meals by respondents, shading, facial hair or glasses on their faces.
  26. 26. Quantum Insight 26 DATA ANALYSIS The recording of each respondent is subjected to two types of analysis: emotional and commitment. At the beginning, we perform a classification of emotions in individuals through a set of algorithms implemented in Quantum Sense. The end result of this process is to iden- tify the emotion that obtains the highest score in a given period of time. Next, an aggregation of individual results is performed - for the whole group of respondents, a per- centage is calculated of specific emotions in the time in- terval of 200 milliseconds. The testing process includes situations in which the ca- mera may “freeze” on the respondent, or instead of the- ir own image they may produce pictures of, e.g. an actor, model, etc. Were such recordings analyzed, it would gre- atly upset the results. With the aim of processing only valid sessions, we have created a special platform to mark „suspicious sessions”. When the system detects such a session, it sends an alert and stores it in a special di- rectory. These recordings are subsequently analyzed by an expert. This guarantees an analysis of only the correct and selected sessions. These safeguards guarantee that the results obtained are reliable. EMOTIONAL INVOLVEMENT The Quantum Insight platform also allows to determine the emotional involvement of a respondent. This parameter indicates the extent to which the material presented will, in total, elicit any emotions (both as a percentage content in the material and as a timing diagram).
  27. 27. Quantum Insight 27 FOCUSING Using Quantum Sense, besides classifying emotions, we are able to determine the position of the head of a subject, relative to the camera and the displayed material. Doshi (La- boratory for Intelligent and Safe Automobiles, University of California) has examined how the estimation of the posi- tion of the head allows, among other things, to determine what is the level of concentration of an individual. Based on scientific reports, our own experiments and the know- ledge of the human muscular system, we have learned to measure the concentration of the viewer in the course of a test. Focus (concentration) is basically about the identi- fication of the respondent’s interest. The research by the Technopole Brest-Iroise institute shows that the position of the head varies, depending on the degree of concentration (Ba SO, 2011). We also added a parameter allowing to deter- mine whether the viewer looks at the displayed material in the course of the study, which may indicate, among other things, when they start to get bored or irritated while wat- ching a video, and which moments of the material are the most interesting for them. PITCH ROLL YAW
  28. 28. Quantum Insight 28 PRESENTATION OF RESULTS The aggregated and analyzed quantitative data are placed on a chart in the form of curves. These curves represent the emotions experienced throughout the research group. Why the whole group, instead of user separately? The- re are two reasons. First, we follow the principles of ethics and data protection - the survey is anonymous and guaran- tees privacy for every user. Secondly, we apply the achie- vements of psychology and statistics - we know that each of us is a unique individual. We are aware of the fact that the subjects will exhibit different reactions to the mate- rial viewed. In addition, we keep in mind the „white noise” that appears on the records - respondents moving whi- le watching the material, yawning, eating meals, etc. To reduce the impact of these factors, we aggregate the emo- tional responses of all respondents to one common result. Thanks to this, the final result is objective. The data obtained as described above is presented in the form of: a line distribution chart of emotions in time, the film integrated with a graph and a table. fig.6. A chart displaying the results of emotional involvement, obtained from the study of advertising.
  29. 29. Quantum Insight 29 Each displays how long the viewers were expressing positive emotions, how long for the negative emotions, and for how long the material did not elicit any emotions. In each case the result is also displayed in the form of a na- tural number, specifying the level of emotional involvement. It is possible to benchmark within the survey conducted by the platform user, or / and within the market segment of the advertised product and the overall comparison of all the ads contained in the Quantum Insight database. The graph indicates with separate colors the moment of exposure of the brand name. The user has the ability to see what exact emotions the audience felt during the presentation. The Quantum Insight platform allows to filter the results. Each bit of information may be viewed, among other things, divided into age groups, sex, place of residence and origin or occupation of the subjects.
  30. 30. Quantum Insight 30 QUANTUM INSIGHT FUNCTIONALITY In addition to the primary function, which is to provide the analysis of the emotions of the given material, the QuantumInsightplatformprovidesvaluableinformationonthe entire study, such as: • whether an ad carries an appropriate emotional load, • which scenes within an ad evoke the strongest emotions, • when customers lose interest in an ad, • how the presentation of the logo of the brand influences the emotional reactions, • how to optimally and effectively shorten the adverti- sing spot, • how to design the future ads. The numerous technological advantages of Quantum Insight include: • the whole process is based on a web browser, • intuitive setting and analysis of results, • the option to change the scale of a diagram, • creation and comparison of video variants, • a  transparent and comprehensive interpretation of results (percentage of respondents who have experien- ced a particular emotion, the average number of occur- rences of emotions and their average duration), • advanced filtering results with the option of comparing selected recordings, • a unified algorithm for calculating the quality index of advertising - Insight Score, • exporting the results in the form of video, infographics, and CSV file, • advertising benchmarking - enables to compare the per- centage advantage of the analyzed ad over other ads, • the option to add declarative questions or select and con- figure them in a simple way, thanks to a built-in bank of certified declarative questions, • the ability to automatically order a panel of respondents (and adaptation of the research group), • platform and tool operates in two languages (English and Polish).
  31. 31. THE FINAL WORD 31 THE FINAL WORD Within 18 months of setting up the Quantum Insight plat- form, the Quantum Lab services have been used by 24 customers. Our team of experts analyzed 804 hours of YouTube videos (more than 100 million video frames). The current number of respondents is 100908 people. So far, we have examined 450 advertising campaigns and 600 video materials. With decades of work by various research units and, in large part, our own experts, we are able to continually improve our technology and equip it with new compo- nents that will be able to identify other factors of human biology. As a result, it has the potential to exist in com- mercial electronics, ‚intelligent’ transport systems, safety and security, electronic assistants, as an aid in the study of customer satisfaction, an element of decision support systems for courtrooms, in employee recruitment and much more. In general, wherever there is a need for fast and automatic interpretation of human state / behavior. We hope it will become a permanent part of our lives, hel- ping and serving us.
  32. 32. AWARDS 32 WHERE WE APPEARED? WHO TALKED ABOUT US? WHAT HAVE WE WON? AWARDS
  33. 33. Bibliography 33 • McDuff, D., Kaliouby, R., Senechal, T., Demirdjian, D., and Pi- card, R. (2014). Automatic measurement of ad preferences from facial responses gathered over the internet. Image and Vision Computing, 32, 630-640. • Cohen, N. Sebe, A. Garg, L. Chen, and T.S. Huang, (2003) Fa- cial expression recognition from video sequences: Temporal and static modeling. Computer Vision and Image Understan- ding, 91(1- 2):160–187. • Cohn, J., Ying-li Tian, B., Kanade, T., (2000). Recognizing ac- tion units for facial expression analysis. In 2000 IEEE Com- puter Society Conference on Computer Vision and Pattern Recognition (CVPR’00) - Volume 1, June 2000. • Cowie, R., Douglas-Cowie, E., Tsapatsoulis, N., Votsis, G., Kollias, S., Fellenz, W., Taylor, J.G, (2001) Emotion recogni- tion in human-computer interaction. Signal Processing Ma- gazine, IEEE, Volume: 18, Issue: 1, Jan 2001. Pages: 32 – 80 • Darwin, C. (1872). The Expression of the Emotions in Man and Animals. London: John Murray. • Ekman, P. (2001). Smiling. In Blakemore, C. & Jennett, S. (Eds.), Oxfors Companion to the Body. London: Oxford Uni- versity Press. • Ekman, P. (2009). Become Versed in Reading Faces. Entre- preneur. Retrieved from http://www.entrepreneur.com/ar- ticle/200934. • Ekman, P. (2009). Lie Catching and Micro Expressions. In Martin, C. (Ed.), The Philosophy of Deception. New York, NY: Oxford University Press • Ekman, P. (2010). Darwin’s Compassionate View of Human Nature. JAMA, 303(6), 557-558. • Barlett, M., Littlewort, G., Fasel, I., Movellan. J., (2003). Real time face detection and facial expression recognition: Deve- lopment and applications to human computer interaction. In Proceeding of the 2003 Converence on Computer Vision and Pattern Recognition Workshop. • Bența Kuderna-Iulian and Vaida Mircea-Florin, (2015) Towards real-life facial expression recognition systems, Advances in Electrical and Computer Engineering, vol. 15(2), pp. 93–102. • Berger, J., Milkman, K. (2012). What Makes online Content Vi- ral? Journal of Marketing Research, Vol. 49, No. 2, pp. 192-205. • Bible, E. (2009). Smiles and frowns are innate, not learned. Retrieved from San Francisco State University: http://www. sfsu.edu/news/2009/spring/1.html • Bradley, M. M., Greenwald, M. K., Petry, M. C., Lang, P. J. (1992). Remembering pictures: Pleasure and arousal in me- mory. Journal of Experimental Psychology: Learning, Me- mory, & Cognition 18 (2): 379–390. • Brockmole, J., Henderson, J., (2005), Prioritization of New Objects in Real-World Scenes: Evidence From Eye Movements, Journal of Experimental Psychology: Human Perception and Performance 2005, Vol. 31, No. 5, 857–868 • Brown, M., (2009). Should My Advertising Stimulate an Emo- tional Response? Millward Brown: Knowledge Point. Bibliography
  34. 34. Bibliography 34 • Goleman, Daniel (1995). Emotional intelligence. New York: Bantam Books. • Graves, J., Robinson, J., II (1976) Proxemic Behavior as a Func- tion of Inconsistent Verbal and Nonverbal Messages. Journal of Counseling Psychology, Vol. 23, No. 4, 333-338 • Kaliouby. P., Kaliouby, R., (2003). Real time facial expression recognition in video using support vector machines. In ICMI ’03: Proceedings of the 5th international conference on Mul- timodal interfaces, pages 258–264, New York, NY, USA, ACM. • Kemeney, M. E., Foltz, C., Cavanagh, J. F., Cullen, M., Gie- se-Davis, J., Jennings, P., Ekman, P. (2012). Contemplative emotion training reduces negative emotional behaviorand promotes procial responses. Emotion, 12(2), 338-350. • McDuff, D., El Kalioubi, R., Cohn, J. F., & Picard, R. (Accep- ted with revisions). Predicting ad liking and purchase intent: Large-scale analysis of facial responses to ads. IEEE Trans- actions on Affective Computing. • Pantic, M., Rothkrantz, L., (2000). Automatic analysis of fa- cial expressions: the state of the art. IEEE Trans. PAMI, 22(12): 1424–1445. • TenHouten,W.D. (2007). General Theory of Emotions and So- cial Life. Routledge. • Teixeira, T., Wedel, M., and Pieters, R. (2012). Emotion-indu- ced engagement in internet video ads. Journal of Marketing Research, Vol. 49, No. 2, pp. 144-159. • Tsiamyrtzis, P., Dowdall, J., Shastri, D., Pavlidis, I. T., Frank, M. G., & Ekman, P. (2007). Imaging Facial Physiology for the Detection of Deceit. Internetional Journal of Computer Vi- sion, 71(2), 197-214. • Ekman, P. (2016). What Scientists Who Study Emotion Agree About 11(1), 31-34. • Ekman, P., Friesen, W. V. (2003). Unmasking the Face. Cam- bridge: Malor Books., p.169 • Ekman, P., Friesen, W. V., (1978) W. V. Facial Action Coding System: A Technique for Measurement of Facial Movement. Consulting Psychologists Press Palo Alto, California. • Ekman, P., Friesen, W., Hager, J., (2002), Facial action coding system, A Human Face. • Ekman, P., Friesen, W.V. (1971). Constants across cultures in the face and emotion. Journal of Personality and Social Psy- chology 17 (2): • Ekman, P., O’Sullivan, M. (2006). From Flawed Self-Asses- sment to Blatant Whoppers: The Utility of Voluntary and In- voluntary Behavior in Detecting Deception. Behavioral Scien- ce and the Law, 24(5), 673-686. • Ekman, Paul (1999). Basic Emotions. In T. Dalgleish and M. Po- wer. Handbook of Cognition and Emotion. Sussex, UK: John Wiley & Sons, Ltd. • Ekman, Pl, (1992). Facial Expressions of Emotion: An Old Controversy and New Findings”. Philosophical Transactions of the Royal Society (London) B335: 63–69 • Facial Action Coding System (FACS) and the FACS Manual”. Face-and-emotion.com. Retrieved 2013-10-26. • Frank, M. G., Matsumoto, D., Ekman, P. Sinuk, K., Kurylo, A. (2008). Improving the Ability to Recognize Micro Expressions of Emotion. In Review. • Godavarthy, Sridhar. (2011). „Microexpression spotting in vi- deo using optical strain”. Web. Retrieved 15 June 2011.
  35. 35. Bibliography 35

×