Our ears are the original nexus of information security. The environments we're in are constantly streaming valuable information to us. All we have to do is listen properly. "Let he who has ears" and all that. Join me as we explore the fascinating world of audio security.

1. The sound of evil
Wes Widner
@kai5263499

2. Meanwhile, in 2013...
vs

3. tl;dr Alexa won
● Google Glass was discontinued in 2015
● “10s of millions” Alexa units sold for Christmas in 2017
● Amazon owns 70% of the smart speaker market
● Amazon is hiring more developers for Alexa than Google
is hiring for everything
● Amazon is hiring 4x more for Alexa than Apple is for Siri

5. What’s exciting
about audio
interfaces?
● The future is hearables
● Serves as a mental enhancement
● Conversational computing
● Personal digital assistants
● Everlasting life
● Burglar alarm
● Health tracking
● Mind reading
● Less ads?

6. This isn’t a return to radio
● The personal computer
revolution made TVs
interactive
● Hearables make our
audio space interactive

8. How do voice assistants work?

9. Speech to text is old
● Tetris creator Alexey Pajitnov
worked on a voice assistant for the
KGB
● Dragon Naturally Speaking is over
21 years old

10. Insights to
something new

11. Plot twist, the cloud yells back..

12. How speech to text works
● Audio is sampled and fingerprinted
● Multiple fingerprints along
a sliding window guard against noise
● Overlap between image and audio
classification

13. Dejavu https://github.com/worldveil/dejavu

17. The sound of silence
● Voice assistants need to also tackle the far field voice
problem
● Cell phone makers try to solve this problem in reverse
● Everything is noise at some point
● Silence is surprisingly noisy
● Breathing is 10db, a whisper is 30db, conversation is 60db
● Absolute silence drives us insane

18. What’s sent to the cloud?
● Raw audio is not continually piped to the cloud
● A stream of fingerprints
● Over a secured connection
● Skills are also required to use secured connections

19. Once you have the text, then what?
● Deciphering text from
speech is just the first
challenge
● Natural language
processing is the second
● The goal is a flexible
taxonomy
Speech to text
Text to intent
Intent to action

20. Conversation as UX
● Programming languages help us express machine code
● Apple’s Knowledge Navigator came out in 1987
● Clippy was annoyingly pendandic
● Google’s assistant can now make a reservation for you
● Slack apps are a natural fit for voice user interfaces

21. Alexa, tell Cloud to eat my shorts!
(Skill) (Intent) (Slot)(Hotword)
Speech Synthesis
Markup Language
(Action)

22. Audio attack vectors

23. How voice assistants go bad
● Alexa is an Android
● Side channel commands
● Over privileged
● Data capture/exfiltration
● Teaching users bad infosec habits

25. Roll your own voice assistant
● Lots of reasons to roll
your own voice
assistant
● Learn how sound is
processed
● Voice style transfer
● Keep all data on-prem
● Sound event detection

27. Some big challenges to solve
● Sub and super-sonic side channels
● Voice identity
● Voice authorization
● App identification
● App authorization
● Conveying sensitive information

28. Generative Adversarial Network
● One network generates, another network evaluates
● This is how voice style transfer works
● GAN pipelines are currently very brittle

31. Model development
● Speech
○ LJ Speech - LibriVox
○ Blizzard Challenge 2017
○ Ryerson Audio-Visual
Database of Emotional
Speech and Song
● Sounds
○ Urban Sound Dataset
○ Google’s AudioSet
● Big need for portable models

32. Biggest attack vector is social engineering
● Personality is the UX of audio
● Concern over how children interact with Alexa
● Cortana turned into a Hitler loving sex robot
● There’s an ongoing debate over the gender of voice assistants
● Almost half of US cell phone calls will be scams next year
● Example: Enkeltrick or grandparent trick est 1968
● Another fun example is Soupy Sales in 1965

33. Weaponized personality
● Audio interfaces are
increasingly being used for
counseling
● Sarcasm as a service
● Bottom line, don’t trust the
voice in your head

34. How to steal a voice
● Legitimate uses
include Roger Ebert
regaining his “voice”
in 2010 with the help
of CereProc
● Voice cloning as a
service - Lyrebird
and Adobe VoCo

35. Gone in 3.7 seconds

36. Worth following
Vijay Balasubramaniyan
Pindrop Security
Brian Roemmele
Futurist

37. Speech Synthesis
Markup Language
Recap

38. Thanks for coming!
es Widner
kai5263499
es@manwe.io
tps://github.com/kai5263499/audio-security-awesome