Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Processing Natural Language in Robotics Applications<br />01001000 01100101 01101100 01101100 01101111 00100000 01010111 0...
Related work<br />Wubble Voice Command Demo (Gazebo Simulation)<br />Arizona Robotics Research Group - University of Arizo...
Related work<br />Humanoid robot speech recognition and object tracking<br />http://www.youtube.com/watch?v=0jW9LgtiiM8<br />
Goals<br />Implement sentence recognition in text form (console/gui)<br />Provided some basis for speech processing <br />...
Existing tools<br />Possible python library implementation<br />Open Rave<br />Text Processing<br />Speech Processing<br /...
Parsing orders<br />Ambiguous interpretations (robot command sentences may suffer less from this)<br />	β€œRobot, make her d...
Parts of speech tagging<br />Given a word, what is its part of speech?<br />π΄π‘Ÿπ‘”π‘€π‘Žπ‘₯: 𝑃(π‘‡π‘Žπ‘”π‘ |π‘Šπ‘œπ‘Ÿπ‘‘π‘ )<br />Β <br />
Parts of speech tagging<br />π΄π‘Ÿπ‘”π‘€π‘Žπ‘₯: 𝑃(π‘‡π‘Žπ‘”π‘ |π‘Šπ‘œπ‘Ÿπ‘‘π‘ )<br />Given a known corpus, maybe it’s easier to predict a word given a ...
Parts of speech tagging<br />π΄π‘Ÿπ‘”π‘€π‘Žπ‘₯: 𝑃(π‘‡π‘Žπ‘”π‘ |π‘Šπ‘œπ‘Ÿπ‘‘π‘ )<br />Bayes: 𝑃𝐴𝐡=𝑃𝐡𝐴𝑃(𝐴)𝑃(𝐡)<br />π΄π‘Ÿπ‘”π‘€π‘Žπ‘₯:Β π‘ƒπ‘Šπ‘œπ‘Ÿπ‘‘π‘ π‘‡π‘Žπ‘”π‘ π‘ƒ(π‘‡π‘Žπ‘”π‘ )𝑃(π‘Šπ‘œπ‘Ÿπ‘‘π‘ )<br /...
Parts of speech tagging<br />π΄π‘Ÿπ‘”π‘€π‘Žπ‘₯: 𝑃(π‘‡π‘Žπ‘”π‘ |π‘Šπ‘œπ‘Ÿπ‘‘π‘ )<br />Bayes: 𝑃𝐴𝐡=𝑃𝐡𝐴𝑃(𝐴)𝑃(𝐡)<br />π΄π‘Ÿπ‘”π‘€π‘Žπ‘₯:π‘ƒπ‘Šπ‘œπ‘Ÿπ‘‘π‘ π‘‡π‘Žπ‘”π‘ π‘ƒπ‘‡π‘Žπ‘”π‘ <br />Β <br />
Calculating ArgMax and likely tags<br />Speech and Language Processing - Jurafsky and Martin <br />
Further work<br />Once actions and their targets are mapped, generate actions<br />Directly executing OpenRave commands in...
Questions?<br />Thank you!<br />
Upcoming SlideShare
Loading in …5
×

NLP for Robotics

2,959 views

Published on

  • Login to see the comments

NLP for Robotics

  1. 1. Processing Natural Language in Robotics Applications<br />01001000 01100101 01101100 01101100 01101111 00100000 01010111 01101111 01110010 01101100 01100100 00100001 00100000<br />Alan Shen<br />
  2. 2. Related work<br />Wubble Voice Command Demo (Gazebo Simulation)<br />Arizona Robotics Research Group - University of Arizona<br />http://www.youtube.com/watch?v=atB9mh6u1Ng<br />http://ua-ros-pkg.googlecode.com<br />
  3. 3. Related work<br />Humanoid robot speech recognition and object tracking<br />http://www.youtube.com/watch?v=0jW9LgtiiM8<br />
  4. 4. Goals<br />Implement sentence recognition in text form (console/gui)<br />Provided some basis for speech processing <br />β€œPick up the blue cup”<br />Pick up the blue cup<br />
  5. 5. Existing tools<br />Possible python library implementation<br />Open Rave<br />Text Processing<br />Speech Processing<br />Robot Action<br />Prairie Dog Libraries<br />Python Tagging Libraries <br />(eg: NLTK)<br />
  6. 6. Parsing orders<br />Ambiguous interpretations (robot command sentences may suffer less from this)<br /> β€œRobot, make her duck”<br /> β€œGet the elevator”<br />Mapping verbs to targets: β€œFollow that person” β€œGet in the elevator” β€œPick up that object”<br />
  7. 7. Parts of speech tagging<br />Given a word, what is its part of speech?<br />π΄π‘Ÿπ‘”π‘€π‘Žπ‘₯: 𝑃(π‘‡π‘Žπ‘”π‘ |π‘Šπ‘œπ‘Ÿπ‘‘π‘ )<br />Β <br />
  8. 8. Parts of speech tagging<br />π΄π‘Ÿπ‘”π‘€π‘Žπ‘₯: 𝑃(π‘‡π‘Žπ‘”π‘ |π‘Šπ‘œπ‘Ÿπ‘‘π‘ )<br />Given a known corpus, maybe it’s easier to predict a word given a tag:<br />Bayes: 𝑃𝐴𝐡=𝑃𝐡𝐴𝑃(𝐴)𝑃(𝐡)<br />Β <br />
  9. 9. Parts of speech tagging<br />π΄π‘Ÿπ‘”π‘€π‘Žπ‘₯: 𝑃(π‘‡π‘Žπ‘”π‘ |π‘Šπ‘œπ‘Ÿπ‘‘π‘ )<br />Bayes: 𝑃𝐴𝐡=𝑃𝐡𝐴𝑃(𝐴)𝑃(𝐡)<br />π΄π‘Ÿπ‘”π‘€π‘Žπ‘₯:Β π‘ƒπ‘Šπ‘œπ‘Ÿπ‘‘π‘ π‘‡π‘Žπ‘”π‘ π‘ƒ(π‘‡π‘Žπ‘”π‘ )𝑃(π‘Šπ‘œπ‘Ÿπ‘‘π‘ )<br />Don’t need to normalize…<br />Β <br />
  10. 10. Parts of speech tagging<br />π΄π‘Ÿπ‘”π‘€π‘Žπ‘₯: 𝑃(π‘‡π‘Žπ‘”π‘ |π‘Šπ‘œπ‘Ÿπ‘‘π‘ )<br />Bayes: 𝑃𝐴𝐡=𝑃𝐡𝐴𝑃(𝐴)𝑃(𝐡)<br />π΄π‘Ÿπ‘”π‘€π‘Žπ‘₯:π‘ƒπ‘Šπ‘œπ‘Ÿπ‘‘π‘ π‘‡π‘Žπ‘”π‘ π‘ƒπ‘‡π‘Žπ‘”π‘ <br />Β <br />
  11. 11. Calculating ArgMax and likely tags<br />Speech and Language Processing - Jurafsky and Martin <br />
  12. 12. Further work<br />Once actions and their targets are mapped, generate actions<br />Directly executing OpenRave commands in sequence?<br />Commands ignored if OpenRave is busy<br />Determine method of controllingPrairieDog movement<br />Need to ensure that text command interfaceis compatible with both arm and wheel controls<br />Populate objects in robot’s word dictionary<br />Eg: block, door, elevator, person<br />
  13. 13. Questions?<br />Thank you!<br />

Γ—