Racing with Artificial Intelligence

1. Wednesday, July 31, 2019 1. Intro & Activity Update 2. Community Open Mic 3. Alex Barbosa Coqueiro - Public Sector SA Manager @ AWS Canada: “Racing with AI” 4. Networking 1 ServerlessToronto.org Meetup Agenda

2. Why we do what we do? 2 Serverless is New Agile Serverless Dev (gluing other people’s APIs and managed services) We're obsessed helping Startups (and creating meaningful MVPs & products) We build bridges between Serverless Community (“Dev leg”), and Front-end & Voice- First folks (“UX leg”), and empower UX developers Achieve agility NOT by “sprinting” faster (like in Scrum), but by working smarter (by using bigger building blocks and less Ops)

3. Why AI/ML topics at Serverless meetup? 3 AWS Machine Learning Stack

4. #ServerlessTO Sponsors 4

5. Manning Publications 2019 giveaways: 1. www.manning.com/books/serverless-applications-with-nodejs 2. www.manning.com/livevideo/production-ready-serverless 3. www.manning.com/livevideo/production-ready-serverless 4. www.manning.com/livevideo/serverless-applications-with-AWS 5. www.manning.com/livevideo/serverless-applications-with-AWS 6. www.manning.com/books/serverless-architectures-on-aws 7. www.manning.com/books/http2-in-action 8. www.manning.com/books/event-streams-in-action 9. www.manning.com/books/the-design-of-everyday-apis 10. www.manning.com/livevideo/graphql-in-motion 11. www.manning.com/books/voice-applications-for-alexa-and-google-assistant 12. www.manning.com/livevideo/machine-learning-for-mere-mortals 13. www.manning.com/books/classic-computer-science-problems-in-python 14. www.manning.com/books/getting-mean-with-mongo-express-angular-and-node 5Check out MEAP program from our Learning Sponsor!

6. Venue Sponsor 6 As Certified B Corporation, Myplanet is purpose-driven and creates benefit for all stakeholders, not just shareholders!

7. Catering Sponsor 7 Also an active Serverless Community member – check out their “Data Pipelines using Serverless Architectures” talk!

8. Devinity 8

9. Devinity… continued 9

10. Community Open Mic 10 10 seconds of freedom to pitch yourself, or your company

11. Feature Talk Racing with AI 11

12. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. Racing with Artificial Intelligence Alex Coqueiro Head of Public Sector Solutions Architecture for Canada, Latin America and Caribbean AWS @alexbcbr

30. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. Different problems require different learning strategies labeled training data Complexityofdecisions Supervised learning Non-labeled training data

32. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. Robocar (Donkey Car Project) Donkey Car Project https://github.com/sunilmallya/donkey/tree/master/sagetrain http://awsrobocar.s3-website-us-east-1.amazonaws.com/ https://github.com/tescal2/donkey

35. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. Different problems require different learning strategies labeled training data Complexityofdecisions Supervised learning Unsupervised learning Reinforcement Learning Non-labeled training data

37. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. • Build machine learning models in Amazon SageMaker • Train, test, and iterate on the track using the AWS DeepRacer 3D racing simulator • Compete in the world’s first global autonomous racing league, to race for prizes and a chance to advance to win the coveted AWS DeepRacer Cup AWS DeepRacer A fully autonomous 1/18th-scale race car designed to help you learn about reinforcement learning through autonomous driving

38. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. Robotic autonomous race car DeepRacer: An exciting way for developers to get hands-on experience with Reinforcement Learning Racing LeagueVirtual simulator, to train and experiment

42. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. Incentivizing centerline behavior 0.1 0.1 0.1 0.1 0.1 0.1 0.1 0.1 S 2 2 2 2 2 2 G = 2 0.1 0.1 0.1 0.1 0.1 0.1 0.1 0.1 REWARD FUNCTION 8.6 9.5 8.5 7.5 6.3 5.0 3.5 1.9 S 10.4 9.4 8.2 6.9 5.4 3.8 G = 2 8.6 9.5 8.5 7.5 6.3 5.0 3.5 1.9 MAX VALUE OF EACH STATE AFTER LOTS OF EXPLORING Discount per step 0.9

47. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. Reinforcement Learning Algorithms Compared Value Approximation Policy Approximation Advantages More stable performance when it works, and tends to converge on global optimum Effective in continuous action spaces, can learn stochastic policies, and faster convergence Disadvantages Difficult to converge if too many (state, action) combinations, slower convergence in general, and can’t learn stochastic properties Typically converges to a local rather than global optimum, high variance in estimating the gradient adversely affects stability, and evaluating a policy is generally inefficient Examples Q-Learning, Deep Q Network, Deep Double Q Network Policy Gradient, Proximal Policy Optimization (PPO)

49. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. Coordinate system and track waypoints OUTER BOUNDARY WAYPOINTS TRACK CENTER WAYPOINTS INNER BOUNDARY WAYPOINTS X Y TRACK WIDTH CAR DIRECTION

50. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. AWS DeepRacer Car Specifications CAR 18th scale 4WD with monster truck chassis CPU Intel Atom™ Processor MEMORY 4GB RAM STORAGE 32GB (expandable) WI-FI 802.11ac CAMERA 4 MP camera with MJPEG DRIVE BATTERY 7.4V/1100mAh lithium polymer COMPUTE BATTERY 13600mAh USB-C PD SENSORS Integrated accelerometer and gyroscope PORTS 4x USB-A, 1x USB-C, 1x Micro-USB, 1x HDMI SOFTWARE Ubuntu OS 16.04.3 LTS, Intel® OpenVINO™ toolkit, ROS Kinetic

51. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. Robotic Arms International Space Station Drones Education Water Home Self-Driving Vehicles Autonomous Walker Rover Robot landscape

52. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. Robotics trends in 2018 Robotics is undergoing fundamental change in collaboration, autonomous mobility, and increasing intelligence Source: IDTechEx • Logistics • Construction • Retail • Hospitality • Healthcare Robots are being put to work every day across many industries • Agriculture • Energy Management • Oil and Gas • Facilities Management • Household chores By 2023, it’s estimated that mobile autonomous robots will emerge as the standard for logistic and fulfillment processes By 2030, 70% of all mobile material handling equipment will be autonomous

53. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. Robotic development cycle 2) Develop robotics application 1) Select robotics software framework 1) Deploy and manage application 3) Test and simulate application New application release and update

54. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. Most widely used software framework for teaching and learning about robotics – over 16 million .deb (Linux Debian) packages downloaded in 2018, a 400% increase since 2014 Founded in Stanford labs over 10 year ago, now managed by the Open Source Robotics Foundation (OSRF) Global open-source community supports two products—Robot Operating System (ROS) and Gazebo ROS A set of software libraries and tools, from drivers to algorithms, that help developers build robot applications Gazebo Robust physics engine, high-quality graphics, and programmatic and graphical interfaces to help developers simulate robots Robot Operating System (ROS) primer

55. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. Introducing AWS RoboMaker A service that makes it easy for developers to develop, test, and deploy robotics applications, as well as build intelligent robotics functions using cloud services

57. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. AWS RoboMaker Sample Robot Applications Hello World Navigation and Person Recognition Voice Commands Robot Monitoring Object- following using RL Self- driving using RL

58. AWS Cloud AWS DeepRacer NAT gateway VPC AWS DeepRacer Models Simulation video Metrics AWS DeepRacer Simulation Architecture

59. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. ROS Msg Node Stored File ROS Nodes Model Optimizer Video M-JPEG Web Server Video Inference Results Web Server Publisher Autonomous Drive Control Node Optimized Model Media engine Camera Model Inference engine Manual Drive Navigation Node Servo & Motor AWS DeepRacer Software Architecture

63. Thank you!

64. Future Talks 2019 1

65. Upcoming Serverless Toronto Meetups 2 1. August 12, 2019: 1. A look at Google Cloud Functions – Matt Welke // 2. Firebase Crash Course – Kudz Murefu 2. September 12, 2019: Serverless Design Patterns and Best Practices – Mike Apted, Startup SA at AWS 3. October, 2019: Serverless CI/CD pipelines with AWS CodePipeline and CodeBuild, vs CircleCI, vs Travis, vs Seed – Frank Wang & Jay V 4. November or December: Serverless Heroes (authors of “Serverless Applications with Node.js”) Slobodan Stojanović & Aleksandar Simović will be here! 5. December or January, 2020: re:Invent recap – Jonathan Dion, Senior Technical Evangelist at AWS

Racing with Artificial Intelligence

Daniel Zivkovic

Racing with Artificial Intelligence