Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Control your Voice like a Bene Gesserit

47 views

Published on

This is my talk for the AppDevCon 2018 where I introduce attendees to the wonderful world of Voice User Interfaces. It starts with a brief introduction to Alexa Skills and Google Home Actions and how to write one of them. Then it explains the benefits of having a mobile app implemented with a good architecture.

Published in: Software
  • Be the first to comment

  • Be the first to like this

Control your Voice like a Bene Gesserit

  1. 1. Control your Voice like a Bene Gesserit Jorge D. Ortiz Fuentes @jdortiz
  2. 2. A Canonical Examples Production
  3. 3. @jdortiz Agenda ★ Context ★ Voice User Interfaces 101 ★ Advanced Architecture Brief ★ VUI + Adv. Architecture ★ Lessons Learned
  4. 4. Warning
  5. 5. Sorry!
  6. 6. Put all your devices in non listening mode
  7. 7. Some Context
  8. 8. Living like a 3rd Stage Guild Navigator
  9. 9. @jdortiz The Request ★ Existing App ★ Refactored to an Advanced Architecture ★ Wanted to explore VUI ★ Prototype with minimum cost
  10. 10. Voice User Interfaces 101
  11. 11. Welcome to the Future
  12. 12. @jdortiz Maybe not as good as we wished (YET) ★ “Sorry. I don’t know that one” “Sorry about that. I’m still learning” ★ English, German, French, Japanese (Voice interaction) ★ Private actions
  13. 13. The Contenders Alexa Google Home SiriCortana
  14. 14. – Bene Genesserit “I must not fear. Fear is the mind-killer. Fear is the little-death that brings total obliteration. I will face my fear.”
  15. 15. The Guts Domain 1 Domain 2 Domain 3 Domain n Routing Natural Language Processing Automatic Speech Recognition Text to Speech Voice ➡ Voice Text Domain & Intent & Parameters Text
  16. 16. Domain nDomain n Extend the System JSON ➡ JSON HTTPS Parse Request Do Your Stuff Build Response
  17. 17. @jdortiz Development ★ Node JS library (Alexa Skills & Google Actions) ★ Also Alexa SDK for Java(/Kotlin) ★ Several unofficial ones
  18. 18. @jdortiz var routes = Routes() routes.add(method: .post, uri: “/") { request, response in defer { response.completed() } if let postString = request.postBodyString, let data = postString.data(using:.utf8) { let reply = AlexaServiceResponse(version: "1.0", response: AlexaResponse(outputSpeech: OutputSpeech(text: "Hello. I am your Genie. How can I help you?", type: .plaintext))) let encoder = JSONEncoder() let responseData = try! encoder.encode(reply) let responseString = String(bytes: responseData, encoding: .utf8) !?? "" response.setHeader(.contentType, value: "application/json") response.setBody(string: responseString) } else { response.badRequest(message: "No request data found.") } }
  19. 19. @jdortiz var routes = Routes() routes.add(method: .post, uri: “/") { request, response in defer { response.completed() } if let postString = request.postBodyString, let data = postString.data(using:.utf8) { let reply = GAV1ServiceResponse(speech: "Hello. I am your Genie. How can I help you?", data: gaServiceRequest.originalRequest.data) let encoder = JSONEncoder() let responseData = try! encoder.encode(reply) let responseString = String(bytes: responseData, encoding: .utf8) !?? "" response.setHeader(.contentType, value: "application/json") response.setBody(string: responseString) } else { response.badRequest(message: "No request data found.") } }
  20. 20. OK But… Why do I need a good architecture?
  21. 21. Advanced Architecture
  22. 22. @jdortiz Persistance FW View Network LocationFW Presenter Entity Gateway Clean Architecture Interactor Entity
  23. 23. Advanced Architecture: Mobile App Delegate / Application View (VC) / Activity Presenter Use Case Entity Gateway Connector Tap on phone button User wants to talk to contact Start call if… Get contact details
  24. 24. @jdortiz Injecting Dependencies View Presenter UseCaseFactory Entity Gateway Connector
  25. 25. VUI + Advanced Architecture
  26. 26. Kwisatz Haderach
  27. 27. Advanced Architecture VUI Server UI Presenter Use Case Entity Gateway Connector Reuse Domain Logic Intent StartCall User wants to talk to contact Start call if… Get contact details
  28. 28. Lessons Learned
  29. 29. – Dune, Frank Herbert “Muad'Dib learned rapidly because his first training was in how to learn. And the first lesson of all was the basic trust that he could learn… Muad'Dib knew that every experience carries its lesson.”
  30. 30. The not-so-relevant Technical Stuff
  31. 31. @jdortiz Implementation Details ★ Rebuild your Model (Alexa Skills) ★ Lots of corner cases in Codable ★ Documentation of the JSON requests / responses could be better
  32. 32. @jdortiz var routes = Routes() routes.add(method: .post, uri: “/alexa/“, handler: alexaRequestHandler) routes.add(method: .post, uri: “/gaction/“, handler: gactionRequestHandler) let server = HTTPServer() server.addRoutes(routes) server.serverPort = UInt16(httpPort) do { try server.start() } catch PerfectError.networkError(let err, let msg) { LogFile.debug("Network error thrown: (err) (msg)") }
  33. 33. @jdortiz func gactionRequestHander(request: HTTPRequest, response: HTTPResponse) { defer { response.completed() } LogFile.info("!>> Google Action Service Request received !>>") if let postString = request.postBodyString, let data = postString.data(using:.utf8) { do { let decoder = JSONDecoder() let gaRequest = try decoder.decode(GAV1ServiceRequest.self, from: data) let reply = try gaction.process(serviceRequest: gaRequest) let encoder = JSONEncoder() let responseData = try! encoder.encode(reply) let responseString = String(bytes: responseData, encoding: .utf8) !?? "" LogFile.debug("Response: (responseString)") response.appendBody(string: responseString) response.setHeader(.contentType, value: "application/json") } catch let error as GActionError { response.append(error: error) LogFile.error("Google Action error: (error)”) } catch let error { response.badRequest(message: "Request data is wrong.") LogFile.error("Error decoding request: (error)") } } else { response.badRequest(message: "No request data found.") LogFile.error("No request data found.") } }
  34. 34. @jdortiz let skill = AlexaSkill(id: appId, welcomeMessage: "Welcome to Family Chores. How can I help you?", logger: GenericLogger()) skill.register(controller: RecordTaskAlexaUIController(), forIntent: "RecordTask") let connector = PersonReportAlexaConnector() skill.register(controller: PersonReportAlexaController(), forIntent: "PersonReport")
  35. 35. @jdortiz protocol AlexaUIController { func process(serviceRequest: ASServiceRequest) throws !-> ASServiceResponse } class AlexaSkill { !//… func process(serviceRequest: ASServiceRequest) throws !-> ASServiceResponse { guard serviceRequest.safetyCheck(id: id) else { throw SkillError.wrongSkillId } switch serviceRequest.request.type { case .launch: return serviceRequest.reply(message: welcomeMessage) case let ASRequestType.intent(data: intentData): if let controller = controllerMap[intentData.name] { return try controller.process(serviceRequest: serviceRequest) } else { throw SkillError.undefinedIntent } case .sessionEnd: return serviceRequest.end() } } }
  36. 36. Identity & Flow
  37. 37. @jdortiz Flow ★ Don’t make your users talk their way through ★ Straight from A -> B
  38. 38. But wait! There is concurrency
  39. 39. @jdortiz skill.register(controller: PersonReportAlexaController(), forIntent: "PersonReport")
  40. 40. @jdortiz skill.register(connector = PersonReportAlexaConnector(), forIntent: "PersonReport")
  41. 41. Fast Prototyping
  42. 42. @jdortiz Quality Time ★ Avoid duplication to address both platforms ★ Ignore the DB ★ Reuse and extend use cases
  43. 43. Recap
  44. 44. @jdortiz Key Takeaways ★ Having (/Going to) an Advanced Architecture is key ★ Pros & Cons of own API interface ★ Easy Prototyping ★ Speak what you wish /Use what you wish
  45. 45. Bedankt!
  46. 46. Thank You!
  47. 47. @jdortiz

×