Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Voice assistants
What it might help us with testing and development?
Oleksandr
Khotemskyi
Independent Contractor
Software Developer Engineer in Test
xotabu4.github.io
What is voice assistants?
And how do they work?
Voice assistants in nutshell
• Always listening microphone connected to
voice recognition service
• Service understands di...
TL DR : just
another interface
to communicate
with computer
Sound processing pipeline
Why only now?
• First prototypes had already appeared in
the late sixties
• Unrealistic expectations from users lead to
di...
Usage examples
How sales see use cases
https://voicebot.ai/amazon-echo-alexa-stats/
Extending with own responses
How developers might extend Google Assistant
Components
• Google Assistant - virtual assistant
powered by artificial intelligence.
Can engage in two-way
conversations
...
DialogFlow
• Works as middleware between user intents
and your responses to them
• Parses inputs, extracts parameters,
pre...
Webhook that calls your
server
Limitations
• The communication is one direction at a time.
Kind of “push-to-talk”. Interface limitation
• “Push notificat...
Usage Examples in development
How it might be useful in everyday work
Launching CI/CD jobs/builds
• Voice commands can invoke with HTTP API
(tested with Jenkins/DroneCI)
• Usually client libra...
Working with JIRA tickets
• Reading status of JIRA tickets
• Updating tickets
• Leaving comments
• Getting statuses of spr...
Health/status checks
• Notifying on service death
• Notifying on resources overconsumption
• Custom metrics change notifyi...
Notifying
• Uses push notifications, but requires server
in local network
• Fresh build released
• Branch merged
• Tests f...
Is there advantages?
Why this might be useful?
Improving chatbots
• Voice assistant can greatly improve your
chatbots
• Can replace or cooperate with chatbot
• Response ...
New communication channel
• Voice communication might be easier in
specific cases
• No context switching (No need to switc...
Doubts
• Might be time-wasting
• Additional support for code that not directly
used in project
• Might be hard to provide ...
Materials
• Human-Computer interactions:
https://en.wikipedia.org/wiki/Human%E2%80%93computer_interaction
• What is Google...
Ask your questions now
xotabu4.github.io
QA Fest 2018. Александр Хотемский. Использование голосовых помощников для разработки и тестирования
QA Fest 2018. Александр Хотемский. Использование голосовых помощников для разработки и тестирования
QA Fest 2018. Александр Хотемский. Использование голосовых помощников для разработки и тестирования
QA Fest 2018. Александр Хотемский. Использование голосовых помощников для разработки и тестирования
QA Fest 2018. Александр Хотемский. Использование голосовых помощников для разработки и тестирования
QA Fest 2018. Александр Хотемский. Использование голосовых помощников для разработки и тестирования
QA Fest 2018. Александр Хотемский. Использование голосовых помощников для разработки и тестирования
QA Fest 2018. Александр Хотемский. Использование голосовых помощников для разработки и тестирования
Upcoming SlideShare
Loading in …5
×

QA Fest 2018. Александр Хотемский. Использование голосовых помощников для разработки и тестирования

85 views

Published on

Голосовые помощники могут стать интересной и забавной заменой обычных чат-ботов. Таких помощников можно заставить выполнять практически любые ваши команды и они неплохо распознают голоса. Это приводит к интересным особенностям использования и дает дополнительные каналы информации. Посмотрим зачем это вообще нам нужно, а если окажется что ненужно - мы будем это знать.

Published in: Education
  • Be the first to comment

  • Be the first to like this

QA Fest 2018. Александр Хотемский. Использование голосовых помощников для разработки и тестирования

  1. 1. Voice assistants What it might help us with testing and development?
  2. 2. Oleksandr Khotemskyi Independent Contractor Software Developer Engineer in Test xotabu4.github.io
  3. 3. What is voice assistants? And how do they work?
  4. 4. Voice assistants in nutshell • Always listening microphone connected to voice recognition service • Service understands different languages, and context • Lots of ready to use reactions to user input, and can be extended by custom, own reactions • Made possible due to great progress of voice understanding algorithms
  5. 5. TL DR : just another interface to communicate with computer
  6. 6. Sound processing pipeline
  7. 7. Why only now? • First prototypes had already appeared in the late sixties • Unrealistic expectations from users lead to disappointment and AI winter of the 70s- 80s • Huge performance of modern computers allows to apply machine learning algorithms for natural language processing • Biggest companies in the world started to invest money, and prepared consumer- ready solutions
  8. 8. Usage examples How sales see use cases
  9. 9. https://voicebot.ai/amazon-echo-alexa-stats/
  10. 10. Extending with own responses How developers might extend Google Assistant
  11. 11. Components • Google Assistant - virtual assistant powered by artificial intelligence. Can engage in two-way conversations • Actions on Google is platform to extend Google Assistant with own reactions • DialogFlow - an end-to-end development suite for conversational interfaces (e.g., chatbots, voice- powered apps and devices)
  12. 12. DialogFlow • Works as middleware between user intents and your responses to them • Parses inputs, extracts parameters, prepares entities with needed data • Integrates to different services, has API, and SDKs • Has training mechanism • Supports context, fallbacks, events
  13. 13. Webhook that calls your server
  14. 14. Limitations • The communication is one direction at a time. Kind of “push-to-talk”. Interface limitation • “Push notifications aren't currently supported on voice-activated speakers” (September 2018). google-home-notifier package can be used, but requires listening server in local network • Hard to pass complex parameters. Works best with single words and numbers • For best results - visual representation will be needed (smartphone, tv, smart display)
  15. 15. Usage Examples in development How it might be useful in everyday work
  16. 16. Launching CI/CD jobs/builds • Voice commands can invoke with HTTP API (tested with Jenkins/DroneCI) • Usually client libraries exist for most popular CI/CD systems • You will need API key. Be sure you are using it securely • You can pass parameters for jobs using dialogflow
  17. 17. Working with JIRA tickets • Reading status of JIRA tickets • Updating tickets • Leaving comments • Getting statuses of sprints, boards
  18. 18. Health/status checks • Notifying on service death • Notifying on resources overconsumption • Custom metrics change notifying • Asking to check server status • Should be working nice with Google Cloud Platform + Google Cloud Functions
  19. 19. Notifying • Uses push notifications, but requires server in local network • Fresh build released • Branch merged • Tests failed • Code freeze started • Meeting is in 5 minutes! • Whatever notification you want
  20. 20. Is there advantages? Why this might be useful?
  21. 21. Improving chatbots • Voice assistant can greatly improve your chatbots • Can replace or cooperate with chatbot • Response to voice request can be sent via chatbot response
  22. 22. New communication channel • Voice communication might be easier in specific cases • No context switching (No need to switch windows, write commands with hands) • Faster - no need to open laptop, login … • My mom liked it
  23. 23. Doubts • Might be time-wasting • Additional support for code that not directly used in project • Might be hard to provide wide range of useful commands
  24. 24. Materials • Human-Computer interactions: https://en.wikipedia.org/wiki/Human%E2%80%93computer_interaction • What is Google Actions: https://developers.google.com/actions/extending-the-assistant • Starting with Google Actions: https://codelabs.developers.google.com/codelabs/actions-1/#0 • Using in office: https://www.zdnet.com/article/five-ways-voice-assistants-are-going-to-change-the-o • Intonations in voice: https://developers.google.com/actions/reference/ssml
  25. 25. Ask your questions now xotabu4.github.io

×