Amazon Web Services release the new AI services during last re:Invent, here we see a little introduction to them and a simple integration with a Lego robot

1. Daniele Madama
@dmadama
EV3RPI - AWS AI services with fun

2. AWS AI
Introducing Amazon AI
Bringing Powerful Artificial Intelligence To All Developers

3. AWS
Services

4. AWS
Services

5. Lex
It’s what’s
inside Alexa

6. Lex
It’s what’s
inside Alexa

7. Lex
It’s what’s
inside Alexa
Build Natural,
Conversational Interactions
In Voice & Text

8. Lex
It’s what’s
inside Alexa
NLU - Natural Language
Understanding

9. Lex
It’s what’s
inside Alexa
NLU - Natural Language
Understanding
ASR - Automatic Speech
Recognition

10. Lex
It’s what’s
inside Alexa
NLU - Natural Language
Understanding
ASR - Automatic Speech
Recognition
Powered By Deep Learning

11. Components
- Bot
- Intent
- Utterances
- Slot
- Fulfillment

12. Lex
- Integration with Mobile Hub
- You can use Lambda as Fulfillment
- Save, Build (Test) and Publish approach

13. Rekognition
Image Analysis
and Recognition
Deep learning based image recognition
service to search, verify and organize
images

14. Rekognition
Image Analysis
and Recognition
- Object and Scene
Detection
- Facial Analysis
- Face Comparison
- Facial Recognition

15. Rekognition
Image Analysis
and Recognition
- Object and Scene
Detection
- Facial Analysis
- Face Comparison
- Facial Recognition

16. Rekognition
Image Analysis
and Recognition
- Object and Scene
Detection
- Facial Analysis
- Face Comparison
- Facial Recognition
Similarity: 98.0%

17. Rekognition
Image Analysis
and Recognition
- Object and Scene
Detection
- Facial Analysis
- Face Comparison
- Facial Recognition

18. Polly
Lifelike Text To Speech
- 47 voices across 24
languages
- Low latency
- Free to reuse

19. Polly
Quality
- Natural sounding speech
A subjective measure of how close TTS output is to human speech.
- Accurate text processing
Ability of the system to interpret common text formats such as abbreviations, numerical sequences,
homographs etc.
- Today in Las Vegas, NV it's 54°F.
- "We live for the music", live from the Madison Square Garden.
- Highly intelligibile
A measure of how comprehensible speech is.
- ”Peter Piper picked a peck of pickled peppers.”

20. Polly
Quality
- Lexicons
Enables developers to customize the pronunciations of word or phrases
<lexeme>
<grapheme>Kaja</grapheme>
<grapheme>Kaja</grapheme>
<grapheme>Kaja</grapheme>
<phoneme>kaI.@</phoneme>
</lexeme>
- Homographs
Words written identically that have different pronunciation
- Proper Names

21. Polly
Quality
- Text Normalization
Disambiguation of abbreviations, acronyms, units (St. -> street/saint, KM -> kilometers)
- Foreign Words
Use of right pronunciation from a different language (C’est la vie, dèjà vu)
- Slang
Support for common used way of saying (ASAP, LOL, ROTFL)
- Prosody / Intonation contour
Prediction of changes in volume, rate, and pitch

22. Polly
SSML
Speech Synthesis Markup Language
is a W3C recommendation, an XML-based markup language for speech
synthesis applications
<speak>
My name is Kuklinski. It is spelled
<prosody rate=’x-slow’>
<say-as interpret-as=”characters”>Kuklinski</say-as>
</prosody>
</speak>

23. Architecture

24. Raspberry PI 3
- AWS services with Java SDK
- Bluetooth speaker
- USB Webcam for image and audio input
- Send Python scripts to EV3Dev
- Send audio but receive text to handle different
cases

25. EV3Dev
- Receive and exec Python scripts
- Interact with motors and sensors
- It’s only a ARM9 300MHz 64MB
- EV3Dev Debian based distro with kernel
modules for Lego Mindstorm EV3 hardware

26. What was hard
- javax.sound API
- Choice of mic (success)
- Play wav file (fail)
- Bluetooth pair CLI
-
- Debugging Lex

27. TODO
- Use AWS IoT button to start/stop audio
recording (no gui needed)
- Alexa Voice Service integration
- Avoid SSH connection between RaspberryPI and
EV3Dev (what?)
- Replace EV3 with BrickPI?

28. Info
https://github.com/madama/ev3rpi