From Dream socialbot to Multiskill AI Assistant Platform
•
0 likes•34 views
This document summarizes DeepPavlov.ai's journey from developing an Alexa Prize socialbot to building a multiskill AI assistant platform. It discusses transforming the socialbot into an open-source platform with reusable NLP models, conversational skills, and AI assistant distributions. It also outlines DeepPavlov's approach to multiskill orchestration, deployment of conversational skills, and use of NLP frameworks and machine learning platforms.
Recommended
More Related Content
What's hot
What's hot (17)
Similar to From Dream socialbot to Multiskill AI Assistant Platform
Similar to From Dream socialbot to Multiskill AI Assistant Platform (20)
More from Daniel Kornev
More from Daniel Kornev (14)
Recently uploaded
Recently uploaded (20)
From Dream socialbot to Multiskill AI Assistant Platform
- 1. FROM ALEXA PRIZE SOCIALBOT TO MULTISKILL AI ASSISTANT PLATFORM Daniel Kornev | CPO @ DeepPavlov.ai
- 2. DeepPavlov.ai Zero Interaction Cost Wish → Action
- 3. DeepPavlov.ai Little Problem: Poor Usability* * Intelligent Assistants Have Poor Usability, Nielsen Norman Group © 2018
- 4. DeepPavlov.ai Good* Bad Bad Bad Terrible Soon to be great Hard to do Limited by design Super hard to do Needs lot of work UI Technique Status Potential * For Non-native English speakers it’s still a pain Voice Input Natural Language Understanding Voice Output Intelligent Interpretation Agency
- 6. DeepPavlov.ai Bad Bad Hard to do Super hard to do UI Technique Status Potential Natural Language Understanding Intelligent Interpretation
- 7. DeepPavlov.ai Grand challenge: create a socialbot that can engage in a fun, high quality conversation on popular societal topics for 20 minutes and achieve an average rating of at least 4.0/5.0. Alexa Prize 3 Winners: 1.Emora - $500K, 3.8/5.0, 7 min, 32 sec, Emora University 2.Chirpy Cardinal -- $100K, 3.17/5.0, Stanford University 3.Alquist -- 50k$, (2nd in ‘17, 3rd in ‘19, ‘20), Czech Technical University
- 8. 1000+ Commits 800+ PRs ~30 skills 10+ annotators 1 PhD! DeepPavlov.ai
- 10. DeepPavlov.ai THE JOURNEY Turning Alexa Prize Socialbot into a Multiskill AI Platform 500+ Commits 100+ PRs +own KBQA +own classifiers for Topics & Dialog Acts +tutorials
- 11. DeepPavlov.ai
- 12. DeepPavlov.ai
- 13. DeepPavlov.ai ?
- 14. DeepPavlov.ai
- 15. DeepPavlov.ai © Copyright PresentationGO.com Pre-Purchase Post-Purchase Surveys Promotions Campaigns Customer Service Technical Support Product Usage Billing & Payment Account Management Logistics
- 16. DeepPavlov.ai ▪ Open repository of NLP models and pipelines • easy to find and reuse NLP components for development of new skills or extension of existing ▪ Open repository of conversational skills • alternative implementations of the most popular skills ▪ Open hub for AI Assistant distributions • general and domainindustry specific distributions of skill sets
- 17. DeepPavlov.ai Multiskill orchestration Conversa- tionalskills NLP frameworks ML platforms Proprietary Open Source ▪ Multiskill orchestration • DeepPavlov Agent is an engine for conversational skill deployment and orchestration ▪ Conversational skills • DeepPavlov Dream is a collection of pre- build conversational skills and a default distribution package for Dream AI Assistant ▪ NLP frameworks • DeepPavlov Library provides pretrained models and simple declarative approach to build NLP processing pipelines ▪ ML platforms • TensorFlow and PyTorch as backends
- 18. DeepPavlov.ai LIVING IN PANDEMICS MAKES US LONELY HOW TO BUILD MULTISKILL AI ASSISTANT WITH DEEPPAVLOV Wouldn’t it be nice to have a friend to care about us?
- 19. DeepPavlov.ai MEET GERTY 3000 Can help Sam with problems on the Moon Base Sarang? Can entertain Sam? Yes ✔ Yes ✔ Can we emulate it with DeepPavlov DREAM? Yes ✔ Main Question Functionality Analysis
- 20. DeepPavlov.ai Text OR Voice Input TTS (NeMo) Spell Checking NeMo ASR Harvesters Status Chit-Chat (AIML) Emotion BUILT-IN SKILL SELECTOR RULE-BASED RESPONSE SELECTOR
- 21. DeepPavlov.ai services: agent: [..] depends_on: - mongo harvesters_maintenance_skill: [..] mongo: [..] rule_based_response_selector: [..] nemo: [..] depends_on: - agent emotion_classification: [..] program_y: [..] spell_checking: [..] Spell Checking Annotators Emotion Classification Harvesters Status Skill Skills Chit-Chat Skill NeMo ASR & TTS Other Services Rule-Based Response Selector
- 22. DeepPavlov.ai services: agent: [..] depends_on: - mongo harvesters_maintenance_skill: [..] mongo: [..] rule_based_response_selector: [..] nemo: [..] depends_on: - agent emotion_classification: [..] program_y: [..] clone_tts: [..] Annotators Services Groups Skills Response Annotators Depend on groups (e.g., “skills”) Services Are Isolated Limited in what they see in dialog Invoke Agent’s State Manager Can run via HTTP or be Python-based "skills": { "harvesters_maintenance_skill": { "connector": { "protocol": "http", "url": "http://harvesters_maintenance _skill:3002/respond" }, "dialog_formatter": "dp_formatters:ful l_dialog", "response_formatter": "dp_formatters:b ase_skill_formatter", "state_manager_method": "add_hypothesi s", "previous_services": ["annotators"] }, Response Selectors
- 23. DeepPavlov.ai What is (all) harvesters’ status? Intents What is harvester status? Prepare rover for a trip domain.yml intents: - all_statuses_request - status_request [..] - trip_request responses: utter_status_request: - text: "The harvester {harv_id} is {harv_status}.“ [..] nlu.md ## intent:all_statuses_request - What is the harvesters status? - What is the combines status? [..] stories.md ## harv_status + prepare_trip * status_request - utter_status_request stories.md – training for dialogs RASA Configs nlu.md – training for intents & slots domain.yml – basic ontology for skill Simple and easy to use
- 24. DeepPavlov.ai Works with GoBot GoBotWrapper Obtains data from DB Generates NLG stories.md – training for dialogs Tutorial in Google Colab nlu.md – training for intents & slots domain.yml – basic ontology for skill Full sample: use it to train your GoBot and save its output to your Skill GoBotWrapper [..] @app.route("/respond", methods=["POST"]) def respond(): [..] dialogs = request.json["dialogs"] for dialog in dialogs: sentence = dialog['human_utterances'][-1] ['annotations'].get("spelling_preprocessing") [..] uttr_resp, conf = gobot(sentence) response = gobot.getNlg(uttr_resp) responses.append(response) confidences.append(conf) return jsonify(list(zip(responses, confidences)))
- 25. DeepPavlov.ai <?xml version="1.0" encoding="UTF-8"?> <aiml version="2.0"> <category> <pattern>I AM ^ TIRED</pattern> <template> 🙁 <random> <li>Get some sleep<get name="name"/>. You're very tired.</li> <li>Have a rest and be happy! How can I help you?</li> </random> </template> </category> [..] </aiml> </xml> Assistant Profile (Name, Place, etc.) Patterns Greeting scenario Topics Looks up for patterns Dialog Processing Picks random pre-defined response If not sure, confidence is low (0.2) Returns response + confidence
- 26. DeepPavlov.ai DEEPY 3000: DEMO A prototype of a fictional Moonbase A.I. Assistant, inspired by the Moon Movie (2009) made by Duncan Jones > docker-compose up --build > curl --location --request POST 'localhost:4242' --header 'Content- Type: application/json' --data-raw '{"user_id": "name", "payload": “what do I do here?"}'
- 27. DeepPavlov.ai demo.deeppavlov.ai Web Demo @deeppavlov_dream_ai_bot TG Bot github.com/deepmipt/dp-dream-demos Play with demo: Clone and build your own! medium.com/deeppavlov Read us: forum.deeppavlov.ai Talk to us: @DeepPavlovDreamDiscussions TG: @DeepPavlov Twitter/TG: