Recommended
More Related Content
Similar to BRUCE and LUCY | Our first two Chat Bots
Similar to BRUCE and LUCY | Our first two Chat Bots (20)
Recently uploaded
Recently uploaded (20)
BRUCE and LUCY | Our first two Chat Bots
- 1. CHATBOTS AT UNIVERSITY OF CANBERRA BRUCE and LUCY Tom Townsend – Deputy Director, Planning and Architecture Rebecca Armstrong – Deputy Director, Projects and Innovation
- 2. Information Technology Management 28 February 2018 DIGITAL CATAPULT STRATEGY Why bots: Workshops during ITM’s strategic planning process. Chatbot’s clearly address the ‘customer’ pillar and the ‘digitally enabled’ foundation. Customer Community Digitally Enabled Reduce friction Break the rules Pimp the experience Leverage the network Innovation culture Explore supply chains Observation | Insights | Evolution
- 3. Information Technology Management 28 February 2018 WHY DID UC CREATE THE BOTS? • Support students and staff, with a view to streamlining support services across the institution • People want to ask questions in a real way and get a response that’s meaningful for them • By adding chatbots UC are giving students and staff another channel to find assistance quickly and easily • The chatbots create a new streamlined service interface and the opportunity to extend the window for meaningful support
- 4. Information Technology Management 28 February 2018 BRUCE • Staff focused • Q&A based on staff intranet • Internal intranet access BRUCE, UC’s new staff chat bot, is here to help you get easy answers to your day to day questions. You can ask about leave, pay, travel, faculties or for IT help. You may even like to ask Bruce to tell you a joke! Bruce is still learning and if there’s anything you think he should know you can email Bruce. LUCY • Student focused • Based on AskUC database • Authenticated but public LUCY, UC’s new student chat bot, is here to help you get easy answers to your day to day questions. You can ask about enrolment, campus facilities, fees and student support or for IT help. Lucy is still learning and if there’s anything you need help with that she can’t answer she can raise a ticket to the Student Centre for you.
- 6. Information Technology Management 28 February 2018
- 7. Information Technology Management 28 February 2018 BRUCE ARCHITECTURE OVERVIEW • The chatbot API APP is hosted inside University internal IIS server. • Once a staff member starts a chat, Bruce will first look into LUIS for a matching intent. If a matching intent is found, the pre-configured answers will be send in responses for each of the intents. • If no proper intent is found inside LUIS, Bot will use the Q&A maker service to find relevant answers and return them as responses to the staff members question. • Third party API’s can also be called such as a ‘Joke API’ that is triggered by key word.
- 8. Information Technology Management 28 February 2018 BRUCE ARCHITECTURE OVERVIEW WebChat Preferred Channels Bot Framework Connector On Prem Bot API App Staff MS Teams Microsoft Cognitive Services (LUIS) QnA Maker
- 9. Information Technology Management 28 February 2018 BRUCE BY THE NUMBERS 1075 163 1238 0 200 400 600 800 1000 1200 1400 January February Total BRUCE Monthly Usage 231 853 154 BRUCE Services LUIS QandAMaker Joke API Launched 11th of January 2018
- 10. Information Technology Management 28 February 2018 BRUCE FEEDBACK I hope you have a great day Bruce. Let’s chat more later about our wedding arrangements. Not too bad Brucie, keep up the good work Thank you! I had no idea about this. This will help my staff so much.
- 12. Information Technology Management 28 February 2018
- 13. Information Technology Management 28 February 2018 LUCY ARCHITECTURE OVERVIEW • The chatbot API APP is hosted inside University internal IIS server. • LUCY can be accessed through a web chat channel from the AskUC (Dynamics CRM Portal) website. • Once a student starts the chat, LUCY will first look into LUIS for a proper intent. If a matching intent is found, the pre-configured answers will be send in responses for each of the intents. • If no matching intent is found inside LUIS, Bot will use the Q&A maker service to find relevant answers and return them as responses to the Student. • Students can log a case from Bot within Dynamics 365 if no answer is available. • The bot will then issue a Dynamics 365 case number to student which can be used for further interactions with Student Centre or AskUC portal. • The case will be raised and queued in CRM.
- 14. Information Technology Management 28 February 2018 LUCY ARCHITECTURE OVERVIEW WebChat Preferred Channels Bot Framework Connector On Prem Bot API App Student Dynamics 365 Microsoft Cognitive Services (LUIS) QnA Maker
- 15. Information Technology Management 28 February 2018 LUCY BY THE NUMBERS Launched 8th of February 2018 28 215 LUCY Services LUIS QandAMaker 4 239 243 0 50 100 150 200 250 300 January February Total LUCY Monthly Usage
- 16. Information Technology Management 28 February 2018 LUCY FEEDBACK That's ok :) you're awesome :) Just thought I would test how Lucy is going – happy to report that she was spot on answering my 3 questions. It’s definitely looking good and I can see the value in what she (Lucy) is going to be able to provide …. We could wait longer and iron out all the kinks, but I don’t think that’s the right approach. We could be waiting forever to get it exactly right (indeed, that’s the kind of point!).
- 17. OUR APPROACH
- 18. Information Technology Management 28 February 2018 OUR CHATBOT PHILOSOPHY • Keep it domain specific, it’s not an AI assistant. • Resist the urge to over complicate. • Build internal skills with emerging technologies. • It’s about the content not the technology. • Have some fun with it, and learn (while there are still relatively low expectations for the technology) Source: https://chatbotsmagazine.com/why-a-slow-growing-pilot-is-vital-for-chatbot-success-cce7875f93b3
- 19. Information Technology Management 28 February 2018 PROJECT DELIVERY • Resourcing: - Experienced .Net Developer - CRM Developers for CRM Integration for Lucy - Content Writers/Curators (1 x Bruce, 1 x Lucy) Lucy’s FAQs were rewritten and then harvested Bruce FAQs were harvested from MyUC - Testers – anyone we could entice/beg/borrow/steal • Timings: - Bruce Development – approx. three weeks effort in total including content curation - Lucy Development – approx. twenty weeks effort in total including twelve weeks content curation
- 20. Information Technology Management 28 February 2018 PROJECT DELIVERY • Support - Microsoft Webinars and from their Premier Engineers – majority effort on commissioning Azure Cloud – this was our first foray with a production ready application. • Direct Costs - Azure Cloud infrastructure - Content writer • Delivery - Agile • Communications - Lucy – Instagram/Socials, Monitor, Bulletin - Bruce – Bulletin
- 21. Information Technology Management 28 February 2018 CONCERNING OR INAPPROPRIATE QUESTIONS • ITM engaged with health and counselling centre and agreed that the chatbots should not ignore questions with concerning trigger words, nor should it attempt to offer any advice. Instead opting for a neutral referral to the relevant service. • There are approx. 50 trigger words that result in the response shown right. • A subset of keywords also provide a link to UC’s ‘safe and respectful community’ site focused on the Respect.Now.Always campaign.
- 22. Information Technology Management 28 February 2018 LESSONS LEARNT • Strong engagement from the ‘data custodians’ is required - Content curation was Lucy’s biggest challenge, UC brought in a professional writer to review all FAQs • Get your hands dirty and ‘fail fast’, our first prototype chatbot ‘GERTY’ was up and running 2 days (but didn’t last long) - This was critical to understand the limitations and constraints of the platform and shaped much of our decision making moving forward. • Be prepared to iterate regularly - We have deployed and re-deployed BRUCE and LUCY countless times. • Traditional communication channel (email address) available for feedback worked well with approx. 25 changes and updates made to BRUCE in the first 2 days.
- 23. Information Technology Management 28 February 2018 BENEFITS • Limited benefit realisation to date, view to measure: - Reduced enquiries to Student Centre - Reduced enquiries to Service Desk - Increased satisfaction % for support in annual surveys for both staff and students - Self service rate • For continuous improvement, view to measure: - Errors within the bot (confusion triggers) - Retention rate (multiple questions, return users) Additional information: https://venturebeat.com/2017/11/19/7-metrics-for-measuring-chatbot-success/
- 24. Information Technology Management 28 February 2018 NEXT STEPS AT UC • Work with ITMs outsourced provider WIPRO to deepen BAU support. • Further learning about how the bots can transform user experience. • Improve re-training - currently clunky and requires manual intervention. • Consider promoting ‘WhoBot’ after an AD data cleanse. • Explore other opportunities for cognitive services to impact how UC do business. • BRUCE: Explore giving BRUCE the ability to log service desk tickets with a ServiceNow integration. • LUCY: Personalise the bots, ask LUCY when your next assignment is due with an LMS integration.
- 25. Information Technology Management 28 February 2018 OTHER BOT EXAMPLES Department of Social Services – Nadia, Roxy, Oliver, George | https://news.microsoft.com/en-au/features/artificial- intelligence-transforms-even-human-services/ Jetstar – Jessie | https://www.cio.com.au/article/633279/jetstar-chatbot-travellers-rescue-after-mount-agung- eruption/ Open Universities Australia – Unnamed – focus on SMS/iMessage chat function | https://www.campusreview.com.au/2018/02/oua-goes-digital-with-student- chatbot/?utm_medium=email&utm_campaign=CR+BN+Email+- +14+Feb+2018&utm_content=campusreview.com.au%2F2018%2F02%2Foua-goes-digital-with-student- chatbot%2F&utm_source=apneducationalmedia.writemsg.com WeChat – Subscription Bot, Service Bot | https://chatbotsmagazine.com/china-wechat-and-the-origins-of-chatbots- 89c481f15a44