Millions of users use Amazon Alexa with Amazon Echo’s devices for their homes and offices. Alexa was integrated in 10 million BMW and MINI vehicles and starting mid 2018 it will be active in all USA, UK and Germany BMW and MINI. I will explain you: - how Alexa works under the hood, - what are Alexa features and skills, - how to develop your own skill and start using it with your echo dot. In details, I’ll show you the protocol level of smart home devices - how you can build devices that will be controlled using Alexa and integrate into your smart home. Since Alexa is an Amazon service, it is perfectly integrates with AWS Lambdas. I’ll give an overview of serverless architecture advantages. Also I’ll explain how to integrate it with on premise solutions, if you are using other clouds providers or custom data centers. At the end I want to share with you my project of software smart gateway, that emulates smart devices and allows to execute “non-cloud” local Lambdas on smart home devices by triggering them with voice commands from AWS Echo or Google Home.

1. Voice control
with
Amazon Alexa
Tsypuk Roman

2. About me
Sr Software engineer at Lohika, Altran Group
Speaker at conferences, meetups, JUGs
java, kotlin, spring boot, distributed systems,
microservices, security, IoT
Radio HAM, software defined radio
2
Tsypuk
Roman

3. Agenda
1. What is Amazon Alexa?
2. How to develop and apply skills?
3. How does smart home integration works?
4. AWS lambdas
5. Messaging under the hood
6. Smart Gateway
3

4. Alexa
4
Cloud-based voice service available on
tens of millions of devices from Amazon
and third-party device manufacturers.
With Alexa, you build natural voice
experiences interact models
NLP, voice recognition, self educating AI

5. Echo devices
5

6. Alexa features
6

7. BMW & MINI Alexa integration
Today, 8.5 million BMW Group models worldwide are already connected. From mid-2018
Amazon Alexa will be available in all BMW and MINI vehicles in USA, UK and Germany.
7https://www.youtube.com/watch?v=mHCI6lM_Zl8
“By making this step of integrating Alexa
into our models from mid-2018, BMW
and MINI will form a more intrinsic part
of our customers’ digital lifestyles,”
remarked Dieter May, Senior Vice
President Digital Services and Business
Models at the BMW Group.

8. Skills
8
Flash Briefing
defines audio/video/text content added to the
brief of news
“Alexa, tell me news”
Custom
everything is specified by developer
Smart Home
controls IoT devices like lights and thermostats
“Alexa, dim the light”
Video
video content is sent in response
“Alexa change to CNN”

9. Flash Brief’s endpoint requirements
9
● non-password-protected HTTPS endpoint
● either text(4500 characters) or audio content (256kbps mono or
stereo MP3 10 minutes)
● JSON or RSS UTF-8 encoded
● available 24 hours a day/7 days a week
● items order from newest to oldest, based on the date. Alexa may
ignore older items.
● each item in the feed should be unique and should not overlap
with content in another feed item.

10. Skills: Flash Brief
10
ApiGateway
Lambda
1
2
3

11. Demo
Flash Brief
Skill
“JAVA NEWS”
Create Flash brief skill from the scratch
Alexa tells us latest Java world news
“Alexa give me latest news”
11

12. AWS
Lambda
12

13. AWS Lambda
need to provision, manage server
on-demande charged hourly based on compute
capacity , reserved, spot
need DNS, ELB, autoscaling, plus correct
configuration and other techniques
need manual security OS updates
networking knowledge, routes to infrastructure
<Any>
13
Lambda
serverless compute service
charged for 100ms running and number of time is
triggered, no idle time payment
scales in parallel on each trigger
VS
provisioning
charge
scaling
security
interaction
runtime
No infrastructure to maintain
triggered (API GW, IoT, ASK,Kinesis, S3,SNS,
CodeCommit, CloudWatch, CloudFront)
C#, GO, Java, NodeJS, Python code

14. Skills: Custom skill
14
service on
other clouds,
datacenters
2
Lambda
1

15. Router for invocations
15
Lambda
Alexa, ask “big brother” who is the winner ?
Lambda
ARN
ARN
Lambda
ARN
...
invocation

16. Utterance-to-Intent mapping
Alexa, ask “big brother” who is the winner ?
Lambda
utterance
winner
intent 1
intent N
...
● find the winner
● who is lucky
● who is the winner
● get the winner
● winner
utterance
16

17. “BIG BRO”
Alexa will choose the winner
from the audience.
“Alexa, who is present?”
“Alexa, get the winner”
Demo
Custom Skill
17

18. IoT
Smart Home
18

19. Alexa in Wireshark
19

20. Sequence diagram: Search devices
UDP: HTTP
http://ip:port/setup.xml
“Alexa,
find my devices”
20
UDP: UPnP broadcast message
M-SEARCH
TCP: HTTP
GET ip:port/setup.xml
TCP: HTTP
RESPONSE
“Starting
discovery”
“I have found
1 new device...”

21. Sequence diagram: Device control
21
“Alexa, turn
meeting room on”
TCP: HTTP SOAP
POST ip:port/control/basicevent1
TCP: HTTP SOAP
RESPONSE
“OK”

22. WE ARE ENGINEERS
22

23. Demo
23
Kitchen
Room
Hall
ESP-12
light control

24. End-to-End: How it works
24
Lambda
1

25. Why not to create Local Lambdas for Alexa?
25

26. Demo
Smart
Gateway
26

27. Links
1. BMW annonce of Alexa devices promo
https://www.youtube.com/watch?v=mHCI6lM_Zl8
2. Flash Brief requirements
https://developer.amazon.com/docs/flashbriefing/flash-
briefing-skill-api-feed-reference.html
3. Arduino WEMO switch
https://github.com/kakopappa/arduino-esp8266-alexa-
multiple-wemo-switch/blob/master/wemos/Switch.cpp
4. Alexa voice AI with filtering by AWS
https://www.youtube.com/watch?v=2Bazibaz1F8
27

28. Q&A
28