Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Reactive Learning Agents

826 views

Published on

Presented at the Google Developer Group NYC Meetup on 9/12/16 http://www.meetup.com/gdgnyc/events/231371634/

This talk is focused on the design of learning agents using the techniques of reactive machine learning. We explored the difference between software agents (bots), intelligent agents, and learning agents. As the most complex class of agent, a learning agent has a sophisticated internal architecture, which we broke down into different capabilities. Finally, we examined how the techniques from reactive machine learning can allow us to build learning capabilities into our agents.

This talk is not just be about AI concepts, though. It covers a range of pragmatic techniques to aid you in your efforts to implement learning agents. Throughout, we considered where and how we can use external resources like libraries, services, datasets, and even humans to solve sub-problems in the agent design process.

Published in: Software
  • Be the first to comment

Reactive Learning Agents

  1. 1. Reactive Learning Agents Jeff Smith @jeffksmithjr
  2. 2. Bio
  3. 3. Bio
  4. 4. Bio
  5. 5. x.ai @xdotai hello@human.x.ai New York, New York
  6. 6. Designing Agents
  7. 7. Agents
  8. 8. Software Agents
  9. 9. Software Agents Mmmm… Lunch.
  10. 10. Software Agents Sensors Actuators Function
  11. 11. Intelligent Agents
  12. 12. Intelligent Agents Maybe Later
  13. 13. Intelligent Agents Sensors Actuators Knowledge Function
  14. 14. Learning Agents
  15. 15. Learning Agents Not that One!
  16. 16. Learning Agents Sensors Actuators Knowledge Learning Function
  17. 17. Reactive Machine Learning
  18. 18. Reactive
  19. 19. Reactive Systems Responsive Resilient Elastic Message-Driven
  20. 20. Reactive Strategies Replication Containment Supervision
  21. 21. Reactive Machine Learning
  22. 22. Reactive Machine Learning Uncertain DataInfinite Data Laziness Pure Functions Immutable Facts Possible Worlds
  23. 23. Reactive Learning Agents
  24. 24. Learning Agents Sensors Actuators Knowledge Learning Function
  25. 25. Reactive Systems Responsive Resilient Elastic Message-Driven
  26. 26. Reactive
  27. 27. + Learning AgentsReactive
  28. 28. Source of AI Components
  29. 29. Source of AI Components Yours
  30. 30. Source of AI Components Other People’s Yours
  31. 31. Sensors
  32. 32. Sensors •Email •Messaging Platforms •Phone cameras & sensors •Wearables
  33. 33. Sensors •Email •Messaging Platforms •Phone cameras & sensors •Wearables •Your Data Collection Systems
  34. 34. Reactive Data Collection
  35. 35. Reactive Data Collection
  36. 36. Reactive Data Collection
  37. 37. Reactive Data Collection
  38. 38. Reactive Data Collection
  39. 39. Reactive Data Collection
  40. 40. Reactive Data Collection •Replication via Distributed databases •Event-sourcing/immutable fact databases •Uncertainty-aware data models •AP Databases https://skillsmatter.com/skillscasts/7038-lightning-talks-2
  41. 41. Actuators
  42. 42. Actuators •Messaging platforms •Email •Notifications •External API Calls
  43. 43. Actuators •Messaging platforms •Email •Notifications •External API Calls •Robots •Self-driving Cars •Internal API Calls
  44. 44. Reactive Action Sensors Actuators Knowledge Learning Function
  45. 45. Reactive Action Sensors Actuators Knowledge Learning Function
  46. 46. Reactive Action Sensors Actuators Knowledge Learning Function
  47. 47. Reactive Action Sensors Actuators Knowledge Learning Function
  48. 48. Knowledge
  49. 49. Knowledge •External Datasets •Knowledge as a Service APIs
  50. 50. Knowledge •External Datasets •Knowledge as a Service APIs •Your Training corpus •Institutional knowledge
  51. 51. Elixir •Functional Language •Homoiconic Syntax •Concurrency-oriented •Runs on the bEAM (EVM)
  52. 52. User Data user2user1 user3
  53. 53. Reactive Knowledge Systems Adapted from https://github.com/mitchellhenke/circuit_breaker
  54. 54. Reactive Knowledge Systems
  55. 55. Reactive Knowledge Systems
  56. 56. Reactive Knowledge Systems
  57. 57. Reactive Knowledge Systems
  58. 58. Reactive Knowledge Systems
  59. 59. Learning
  60. 60. Learning •Machine Learning as a Service APIs •Open Source Libraries
  61. 61. Learning •Machine Learning as a Service APIs •Open Source Libraries •Your novel Algorithms •Organizational Learning
  62. 62. Reactive Learning
  63. 63. Reactive Learning
  64. 64. Reactive Learning Raw Data Features Model
  65. 65. Reactive Learning Raw Data Features Model
  66. 66. Reactive Learning Raw Data Features Model Raw Data Features Model
  67. 67. Reactive Learning Raw Data Features Model Raw Data Features Model
  68. 68. Reactive Learning Raw Data Features Model Raw Data Features Model
  69. 69. Reactive Learning Raw Data Features Model
  70. 70. Reactive Learning Raw Data Features Model https://youtu.be/B7XYlBaxglU
  71. 71. Agent Function
  72. 72. Agent Function •???
  73. 73. Agent Function •??? •Your Whole system
  74. 74. Reactive Agent Functions
  75. 75. Reactive Agent Functions
  76. 76. Reactive Agent Functions
  77. 77. Reactive Agent Functions
  78. 78. Reactive Agent Functions
  79. 79. Reactive Agent Functions
  80. 80. Reactive Agent Functions
  81. 81. Dialyzer in Action ml_system.ex:3: Function predict/1 has no local return ml_system.ex:6: The call 'Elixir.MLSystem':call_model_b(feature@1::number()) will never return since it differs in the 1st argument from the success typing arguments: (binary()) ml_system.ex:22: Invalid type specification for function 'Elixir.MLSystem':call_model_b/1. The success typing is (binary()) -> binary() ml_system.ex:23: Function call_model_b/1 has no local return
  82. 82. Dialyzer in Action ml_system.ex:24: The call 'Elixir.String':upcase(feature@1::number()) will never return since the success typing is (binary()) -> bitstring() and the contract is (t()) -> t() ml_system.ex:38: Function ensemble/1 will never be called
  83. 83. Testing Agent Functions Tool Tests Dialyzer Types QuickCheck Properties Concuerror Interleavings
  84. 84. Summary
  85. 85. Learning Agents Sensors Actuators Knowledge Learning Function
  86. 86. Reactive Systems Responsive Resilient Elastic Message-Driven
  87. 87. Reactive Machine Learning Uncertain DataInfinite Data Laziness Pure Functions Immutable Facts Possible Worlds
  88. 88. + Learning AgentsReactive
  89. 89. Reactive Machine Learning reactivemachinelearning.com @jeffksmithjr Use the code gdgnycmu for 39% off the book!
  90. 90. x.ai @xdotai hello@human.x.ai New York, New York
  91. 91. x.ai @xdotai hello@human.x.ai New York, New York We’re hiring!
  92. 92. Thanks!
  93. 93. Reactive Learning Agents Jeff Smith @jeffksmithjr

×