Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Demola smart cabs_20120502


Published on

Demola Summer 2012 project for TAMK Social Robotics

Published in: Education
  • Be the first to comment

  • Be the first to like this

Demola smart cabs_20120502

  1. 1. SMART CABS: MACHINES THAT KNOW THEIR DRIVERS Rod Walsh, Petri Murtomaki, and Kimmo Vänni TAMKversion0.1 date 15.03.2012 author RW & KV details Created & first ideas Demola InnoSummer 20120.2 16.03.2012 Rod Walsh Minor improvements0.3 02.05.2012 Rod Walsh Filled out the “complete story” © TAMK, 2012. ALL RIGHTS RESERVED. TAMK CONFIDENTIAL. 1
  2. 2. COMING UP IN THIS SLIDE SET…  Why: Better Performance in Forestry  What: The Human Touch  How: The Demo  Where: The Big Idea  Approach: Approach© TAMK, 2012. ALL RIGHTS RESERVED. TAMK CONFIDENTIAL. 2
  3. 3. BETTER PERFORMANCE IN FORESTRY  The commercial performance of large human-operated machines is largely determined by the performance of the human operator  Today, human operator performance is largely driven by hard external factors, such as training, experience and attitude  Dynamic factors are “left to care for themselves”: such as tiredness, alertness, attentiveness, happiness, etc.  But we want to use technology and human-insight to monitor these soft internal factors  And improve working life, long-term health and commercial productivity© TAMK, 2012. ALL RIGHTS RESERVED. TAMK CONFIDENTIAL. 3
  4. 4. THE HUMAN TOUCH non-contact sensing We will take a look at the emotional state-of-mind of operators using face, sound and posture monitoring technology with pattern recognition Psychology & processing And use our knowledge of these soft internal factors for improvements: state of mind  Happier and lower-stress work (short and long term benefit for the employee)  Better productivity (short and long term benefit for the employer) By:  Dynamically modifying the working environment for the better (short term)  Identifying positive patterns of emotion affect on human Simple changes performance & motivation, and then matching practices, assignments • Music, lighting, airflow, … and environments the patterns (long-term) Pattern Working recognition practices© TAMK, 2012. ALL RIGHTS RESERVED. TAMK CONFIDENTIAL. 4
  5. 5. Examples of “state of mind”THE DEMO • Tiredness • Boredom • Willingness to work • Fear/anxiety Multiple HD webcams, microphones and PrimeSense IR sensors (e.g. • Happiness Kinect) will be arranged to monitor a human “operator” (non-contact sensing) Examples of corrective action:  (For versatility, an “office desk operator” setup is needed. The team may take • Encouragement physical forestry machine mock-ups and closeness-to-reality to higher levels.) • Stimulation A set of “states of mind” that are relevant to machine operator • Pause/end of task performance and wellbeing will be selected • Verify the measurement  Quickly selected emotions at first (for rapid development) & then iterated Examples of Job improvements: Sensor signals are classified for the “states of mind” • Productivity  Classifier(s) will be “trained” and tested. Training and testing will begin with • Volume “acted emotions” and tightly iterated between the pattern recognition and • Errors the pyschology/emotional model. • Motivation for the job Offline: all sensor and analytics data will be logged, to allow discovery • Intervention before of longer-term patterns (such as time of day patterns) problems become critical Real-time: The instantiations state-of-mind is matched against a “task model” and need for corrective action (on the operator) is calculated As determined, corrective action is taken to change the operator’s environment  The effects and affects are logged to determine whether the action succeeded  (The “office desk simulator” can be a PC display simulation, or better…)SEE NEXT SLIDE FOR VISUAL DESCRIPTION© TAMK, 2012. ALL RIGHTS RESERVED. TAMK CONFIDENTIAL. 5
  6. 6. THE DEMO Database: non-contact sensing: state of mind log video, image, audio Pattern sensor logs recognition logged logged offline real-time 7/10 capability state of mind estimation ~7/10 capability Match with task Simulate simple changes 8/10 • Music, lighting, airflow, … minimum © TAMK, 2012. ALL RIGHTS RESERVED. TAMK CONFIDENTIAL. 6
  7. 7. THE BIG PICTURE  For the long-term benefits, the data can be used to change the design of working environments and practices, so…  The demo would be integrated to a larger system (see next slide)  Existing telematics data from the forestry machines can introduced to the common database and analyzed for patterns between operator state of mind and machine behavior (for further insights and causalities)  This is beyond what the team needs to do!  The team’s innovation and excitement decide what is done beyond the core demo© TAMK, 2012. ALL RIGHTS RESERVED. TAMK CONFIDENTIAL. 7
  8. 8. Human impact Database: THE BIG PICTURE on work quality & telematics logproductivity & state of mind log machine sensor logs performance state of mind Machine + Logging telematicsenvironment (exists already) impact on capability human operator capability Match Improvements required corrective action © TAMK, 2012. ALL RIGHTS RESERVED. TAMK CONFIDENTIAL. 8
  9. 9. APPROACH In theory, the team is free to adopt any approach that:  Works well, looks great and receives “ooh” and “wow” sounds  Fits the objectives  Is reusable, extendable and portable (as a whole and as components) Meeting these needs in one go is near impossible, so iteration, communication and sharing are critical – and at high speed! In practice, the support team has some useful experience and advice:  Short design, implementation and demo iterations are the safest and coolest  Stick to technologies which are cross-platform and open (when possible):  E.g. HTML5, OpenNI, Published solutions, etc. as applicable  We will supply USB webcams (inc. microphones) and PrimeSense IR sensors  Code should be runnable on Mac/Win/Linux (Ubuntu is our favorite Linux)  We will workshop together to best use the team’s and the support team’s knowledge© TAMK, 2012. ALL RIGHTS RESERVED. TAMK CONFIDENTIAL. 9
  10. 10. Some support slides…
  11. 11. Together with another awesome project, SMART CAB + we could close the loop on emotional feedback (possible project extension) AFFECTIVE ROBOTS state of mind state of mind Goal estimation Match Emotive“corrective” commandsaction using and like:emotionally-savvy • be happyavatar • welcome 1. Perform the emotion • Reject • cry 2. Perform for the emotion What setting or stage would unlock, actual robot virtual robot 3. Read/write emotion? emphasize or inhibit which affects? body face Full modeled human-like emotion© TAMK, 2012. ALL RIGHTS RESERVED. TAMK CONFIDENTIAL. 11
  12. 12. Human impact non-contact sensing: on work Database: telematics log video, image, audio quality &productivity & Pattern state of mind log machine recognition sensor logs logged performance logged offline “state of mind” Machine + Logging telematics real-timeenvironment (exists already) impact on 7/10 capability human operator state of mind estimation ~7/10 capability Match with task Simulate simple changes Job improvements 8/10 • Music, lighting, airflow, … minimum corrective action © TAMK, 2012. ALL RIGHTS RESERVED. TAMK CONFIDENTIAL. 12
  13. 13. SIMPLE ONE-SLIDER Design of practices &environment telematics © TAMK, 2012. ALL RIGHTS RESERVED. TAMK CONFIDENTIAL. 13