Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

A Clojure Fusion of Symbolic and Data Driven AI

229 views

Published on

The success of deep learning (DL) has again raised the hope for artificial intelligence (AI). As the understanding of the strengths of deep learning deepens, a desire to combine it with other approach emerges. Clojure’s data oriented programming is shown to offer a perfect bridge between symbolic and data driven AI. In this talk, we introduce REP, a domain specific language embedded in Clojure that provides the quickest path to authoring a chat bot. We show how Clojure enables us to combine the strength of symbolic and data driven approach of machine learning to build an AI platform that is easy to use, even for non-programmers.

Published in: Technology
  • Be the first to comment

  • Be the first to like this

A Clojure Fusion of Symbolic and Data Driven AI

  1. 1. A CLOJURE FUSION OF SYMBOLIC AND DATA DRIVEN AI Huahai Yang, Ph.D. Juji, Inc. Nov. 29, 2018 Navigate : Space / Arrow Keys | - Menu | - Fullscreen | - Overview | - Blackout | - Speaker | - HelpM F O B S ?  1 / 34
  2. 2. Psychologist Computer scientist Coding in Clojure since 2012 Huahai Yang  huahaiy  Cofounder & CTO Juji Inc. © 2018 Juji, Inc.  2 / 34
  3. 3. JUJI BUILDS CHATBOT PLATFORM A challenging problem © 2018 Juji, Inc.  3 / 34
  4. 4. JUJI BUILDS CHATBOT PLATFORM It is not hard to pass Turing Test, done in 70s' PARRY: 33 psychiatrists cannot tell it from paranoid patients © 2018 Juji, Inc.  4 / 34
  5. 5. PARRY SAMPLE © 2018 Juji, Inc.  5 / 34
  6. 6. JUJI BUILDS CHATBOT PLATFORM It is harder to build useful chatbots interview people to collect feedback receive visitors to sites screen job candidates check up on trainees © 2018 Juji, Inc.  6 / 34
  7. 7. JUJI BUILDS CHATBOT PLATFORM When used for survey 2X completion rate 76% better quality responses  the whole time i was doing this survey it felt like i was talking to a friend and sharing the same common ground. i loved that i wish it didnt have to end  very dynamic and very fluid conversation you have great quality thanks © 2018 Juji, Inc.  7 / 34
  8. 8. JUJI APPROACH: SYMBOLIC + DATA DRIVEN Symbolic system as the bones Data-driven component as the flesh Done in a Clojure DSL, REP © 2018 Juji, Inc.  8 / 34
  9. 9. AI SUMMER IS BACK Watson Jeopardy beats human AlphaGo beats human Many AI assistants on phone and in home Many commercial products in enterprises © 2018 Juji, Inc.  9 / 34
  10. 10. RISE OF DEEP LEARNING (DL) Recently hugely successful For many: AI = DL © 2018 Juji, Inc.  10 / 34
  11. 11. DL SOLVES PERCEPTION PROBLEM DL maps raw data (pixels, text characters) into: known labels (classification) desirable numbers (regression) fixed length vectors (embedding)  Perception is the organization, identification, and interpretation of sensory information in order to represent and understand the presented information, or the environment. © 2018 Juji, Inc.  11 / 34
  12. 12. PERCEPTION FEELS LIKE INTELLIGENCE Reporter: "How many moves do you see ahead while playing chess?" Capablanca: "Only one, but it's always the right one." © 2018 Juji, Inc.  12 / 34
  13. 13. PERCEPTION IS NOT YET INTELLIGENCE © 2018 Juji, Inc.  13 / 34
  14. 14. INTELLIGENCE CANNOT BE SOLVED WITH DATA ALONE Bottom-up data driven sub-symbolic Top-down goal/hypothesis driven symbolic (human-readable) © 2018 Juji, Inc.  14 / 34
  15. 15. TIME TO BRING BACK SYMBOLIC AI (Semi-)solving perception lays the foundation for symbolic AI The same forces leading to the rise of DL apply to symbolic AI More powerful hardware, help graph search More abundant realistic data, help knowledge base construction Better so ware tools and practices © 2018 Juji, Inc.  15 / 34
  16. 16. TRADE-OFFS Data driven Easy to defeat/abuse by adversaries (e.g. Tay) Hard to debug and bend it to the creator's will By design, unlikely to be fixable Symbolic Easy to build rigid/brittle systems Hard to develop, hard for human to think like machines In principle fixable, in practice, not so easy © 2018 Juji, Inc.  16 / 34
  17. 17. DL IS FOOLED © 2018 Juji, Inc.  17 / 34
  18. 18. GIBBON © 2018 Juji, Inc.  18 / 34
  19. 19. TWO ROADS TO INTEGRATION Implement symbolic phenomenon with sub- symbolic system Mimic brain Not yet practical Symbolic + sub-symbolic Engineer's method Practical today © 2018 Juji, Inc.  19 / 34
  20. 20. SYMBOLIC + DATA DRIVEN Symbolic system as the bones for its potential for growth and adaptability, despite the rigidity DL/ML components as the flesh for its flexibility and ease of development, despite the obscurity © 2018 Juji, Inc.  20 / 34
  21. 21. JUJI ARCHITECTURE © 2018 Juji, Inc.  21 / 34
  22. 22. EDN DATA ALL THE WAY 1 User select chat template 2 User configure chat in GUI 3 Generate script from GUI 4 Chat: script compiles and runs © 2018 Juji, Inc.  22 / 34
  23. 23. DEFTOPIC: THE BUILDING BLOCK Topic: a set of rules 01 (deftopic hello-world ; topic name 02 [] ; parameters 03 04 [] ; trigger 05 ["Hello world!"]) ; action © 2018 Juji, Inc.  23 / 34
  24. 24. PRODUCTION RULE Rule: trigger (if), action (then) and associated followup topics Followup topics are primed when a rule fired 01 [:1 hello hi hey howdy] ; alternative pattern trigger 02 ["Nice to meet you!"] ; action: a string output 03 (talk-about-wheather) ; followup topic invocation © 2018 Juji, Inc.  24 / 34
  25. 25. TOPIC COMPOSITIONS A topic may include rules of other topics 01 (deftopic greetings 02 [] 03 {:include-before [(morning-greetings) 04 (evening-greetings)]} 05 06 [:1 hello hi hey howdy] 07 ["Hello"]) © 2018 Juji, Inc.  25 / 34
  26. 26. PATTERNS Token based regular expressions 01 [I love :1-. pizza] ; sequence with a wild car 02 [love ? [:1- pizza bacon]] ; nested, inner alternativ 03 [:0. "I love pizza"] ; start and string pattern © 2018 Juji, Inc.  26 / 34
  27. 27. ML BASED TAG AND CLASS PATTERNS Tags for annotating text keywords for placeholders of content classes 01 [he #pos/verb dog tree] ; parts of speech tag 02 [she love :phrase/NP] ; noun phrase class 03 [I work at :entity/org] ; organization entity class 04 [did in :entity/duration] ; duration entity class © 2018 Juji, Inc.  27 / 34
  28. 28. DL/ML FOR CLASSIFICATION FUNCTIONS Neural networks are universal function approximator, should be used as such Patterns are and together in a rule Rules are or together, so a topic matches a DNF 01 [programming 02 (input-in-this-category? "self-intro-relevance" 0.7) 03 ["You must be a smart person"] © 2018 Juji, Inc.  28 / 34
  29. 29. DL FOR SIMILARITY BASED MATCHES Calculate similarity using Tensorflow sentence embedding 01 [(> (max-similarity-score 02 ["What does your product cost?" 03 "How much does your product cost?" 04 "What's the price of your product?" 05 "How expensive is your product?"]) 06 0.9)] © 2018 Juji, Inc.  29 / 34
  30. 30. ROLES ML/DL models cover broad cases Symbolic covers specific cases misses by DL/ML detailed refinement 01 [(input-in-this-category? "self-intro-relevance" 02 0.7)] 03 ([programming] 04 "You must be a smart person." 05 06 [art] 07 "I enjoy art too." 08 09 "Thank you for the introduction.") © 2018 Juji, Inc.  30 / 34
  31. 31. META-CIRCULARITY Turn a topic into a function, then use the function in another topic 01 [(create-topic-func 02 custom/why-u-here :extract-why-u-here) 03 "I see, you are here to " 04 (exec-topic-func :extract-why-u-here)] © 2018 Juji, Inc.  31 / 34
  32. 32. AUTOMATIC DIALOG MANAGEMENT REP is a declarative language System pushes topics around Agenda queue Ad-lib queue Exception queue Main stack © 2018 Juji, Inc.  32 / 34
  33. 33. CONCLUSION Symbolic + data driven = practical AI today Clojure is a great choice for doing so Lisp was and still is the language of symbolic AI Data orientation of Clojure makes it easy to integrate both © 2018 Juji, Inc.  33 / 34
  34. 34. Huahai Yang Juji, Inc. https://juji.io © 2018 Juji, Inc.  34 / 34

×