Ubiquitous Machine Learning: Lessons from DeepRL in Robotics and Speech, by Farzaneh S. Fard, Ph.D., Machine Learning Scientist at Fluent.ai
•Download as PPTX, PDF•
0 likes•162 views
"Ubiquitous Machine Learning: Lessons from DeepRL in Robotics and Speech" By Farzaneh S. Fard, Ph.D., Machine Learning Scientist at Fluent.ai Abstract: Machine learning algorithms have been used in wide range of applications. In this talk we will target two applications "Robotics" and "Speech recognition" that have been benefit from ML. In the first part of the talk, "Arbitrated Predictive Actor-Critic" model will be explained that can simulate human target reaching through combining a model-free and a model-based reinforcement learning. The study tests the ability of such a model to control a simulated robotic arm (with two joints) during multiple reaching tasks, including (A) both static and dynamic target properties, (B) absence of visual inputs, © adapting to slowly changing kinematics, and (D) the model also shows how arbitration between the two control systems is explainable. In the second part, some recent techniques and methods that have been used for Speech Recognition as well as some challenges in this field will be mentioned.
Recommended
Recommended
More Related Content
Similar to Ubiquitous Machine Learning: Lessons from DeepRL in Robotics and Speech, by Farzaneh S. Fard, Ph.D., Machine Learning Scientist at Fluent.ai
Similar to Ubiquitous Machine Learning: Lessons from DeepRL in Robotics and Speech, by Farzaneh S. Fard, Ph.D., Machine Learning Scientist at Fluent.ai (20)
More from WiMLDSMontreal
More from WiMLDSMontreal (11)
Recently uploaded
Recently uploaded (20)
Ubiquitous Machine Learning: Lessons from DeepRL in Robotics and Speech, by Farzaneh S. Fard, Ph.D., Machine Learning Scientist at Fluent.ai
- 1. Farzaneh S. Fard Machine Learning Scientist at Fluent.ai March 8th, 2019 – at Element AI 1Farzaneh.fard@fluent.ai
- 2. Ubiquitous Machine Learning • In this talk: • Cognitive Robotics • Speech recognition 2Farzaneh.fard@fluent.ai
- 3. COGNITIVE ROBOTICS • Aspects of cognitive robotics: • Flexible, safe, and adaptive robots • Obtaining better understanding of the brain 3Farzaneh.fard@fluent.ai
- 4. BRAIN CONTROL SYSTEMS • Deliberative planning • Needs the model of the environment to plan ahead • Model-based reinforcement learning / supervised learning • Forward and inverse models • Habitual controller • Learns the reward function by repeating past experiences that are most rewarding • Model-free reinforcement learning • Actor-critic Integrating two controllers? 4Farzaneh.fard@fluent.ai
- 5. PROS & CONS 5 • Deliberative planning • Time consuming • Computationally expensive • Very accurate • Habitual controller • Fast • Less computationally expensive • Needs many examples to train Arbitrated Predictive Actor-Critic (APAC) Farzaneh.fard@fluent.ai
- 6. SIMULATING HUMAN TARGET REACHING • One step mapping • Multi-step reaching • Conditions • Static/ changing kinematics • Static/ changing target 6Farzaneh.fard@fluent.ai
- 7. APAC for TARGET REACHING • Deliberative planning • Habitual controller • With arbitrator 7Farzaneh.fard@fluent.ai
- 8. APAC – MAIN RESULTS • Adaptive to changes both in Kinematics and the environment • Performs as accurate as planning while relying more on habits 8Farzaneh.fard@fluent.ai
- 9. APAC – MAIN RESULTS • Automatically shifting to habitual system from planning after learning 9Farzaneh.fard@fluent.ai
- 10. SPEECH RECOGNITION • Keyword spotting ● Providing users a fully hands-free experience ● Distinguishing wake phrase from ordinary talks ● Always listening - power consumption ● Classification: wake phrase or filler • Challenges • Noise robust • Far field: signal attenuation, reverberation/echo • Small footprint • Speaker identification 10Farzaneh.fard@fluent.ai
- 11. TIME DELAY NEURAL NETWORK 11 Speech input Spectrogram Farzaneh.fard@fluent.ai
- 14. • AI on the edge • On device if on the cloud is not necessary • smart lock • Universal Speech recognition • In any language or multi language • 3 languages: English/German/Korean 14Farzaneh.fard@fluent.ai
- 15. WE ARE HIRING Machine Learning Scientist & Interns www.fluent.ai/careers 15Farzaneh.fard@fluent.ai
Editor's Notes
- ML has wide range of application.. In almost everything you do during a normal day you are using a machine learning algotrithms. Only on our phone we use many of these algorithms, such as pattern recognition, weather forecast, spam checking for your emails, shopping recommenders and many more… Here I will talk briefly about two of these applications. So let’s jump into this.
- Cognitive robotics is a multi disciplinary area that has two major aspects. One aspect is developing robots that can mimic human behavior, which leads us to a more flexible and adaptive robots which are safe to interact and cooperate with human. The other aspect is obtaining better understanding of the brain and try to help people with brain disorders, by studying and testing the theories of how the brain works onto robots and analyze robot's behavior.
- There are many models and algorithms that have been proposed in robotics for each control system. Deliberative planning is time consuming to predict the best action at the time and needs a predefined model. The habitual controller is fast but it is useless in dynamic environment with changing reward functions.
- - The learning happens through integration of these two controller
- Since the main reason to have a brain is being able to move, we chose the target reaching task. We simulated a 2D robot arm and we applied the model to learn the nonlinear function of arm movement to reach the target located in the environment (shown in the picture).
- Integrator could be Kalman-filter.
- We learn a new task with deliberately planning and then it becomes a habit! This work is not similar to Barto&Sutton’s work (supervised AC and Dyna-Q)