Interacting with Intel Edison with Esther Jun Kim Presented at FITC Toronto 2015 More info at www.fitc.ca OVERVIEW This talk will introduce the audience to Intel Edison, Intel’s newest maker platform. It will show how Edison can be transformed into a smart, natural device by teaching it to see and hear while retaining its small form factor. The talk will also briefly describe how to interface Edison with simple supplementary hardware to create a programmable, voice-controlled system. OBJECTIVE Highlight the possibilities of Speech Interaction with Maker Devices. TARGET AUDIENCE Developers, designers, programmers. FIVE THINGS AUDIENCE MEMBERS WILL LEARN What’s Intel Edison. How Language Models Work. Simple Circuits. About LEDs. Voice Interaction.

1. Interacting with Intel Edison
FITC Toronto 2015
Esther Jun Kim (@drejkim)
14 April 2015

2. 2
Video
https://www.youtube.com/watch?v=_VFsdPAoI1g

3. 3
Agenda
Introduction to Edison
Getting Edison to see (with Node.js)
Getting Edison to hear (with Python)
 Interfacing with the GPIO / blinking LEDs
 Speech recognition / speech-activated LEDs
Additional resources

4. Intel Edison
4
Full-fledged computer in a tiny
package
 Dual-core Atom CPU at 500 MHz
 MCU at 100 MHz
 Wi-Fi / Bluetooth LE
 Yocto Linux
 40 GPIOs
Software compatibility
 C, C++, Python, Node.js support
 Intel IoT Developer Kit

5. Mini breakout board
SparkFun blocks
Arduino
breakout board
5

6. SparkFun blocks
6
Stackable blocks that expand Edison’s
functionality while retaining its small
form factor
 14 blocks so far
 Base block – connects peripherals
 GPIO block – provides a friendly
interface to the GPIO pins
Base and GPIO blocks

7. 7
Video
https://www.youtube.com/watch?v=nrwLyjYovw4

8. 8
High-level overview of video streaming on Edison
Edison (server)
• FFmpeg: encodes the
webcam video to MPEG1
• Node.js: Listens for the
encoded video stream via
HTTP
• WebSockets: broadcasts
the video stream to all
connected browsers
Browser (client)
• jsmpeg: decodes the
MPEG1 video
• HTML5 canvas: used to
render and display the
decoded video

9. GitHub
https://github.com/drejkim/edi-cam
9

10. 10
OpenCV
Monitoring w/ motion detection
Eddie3D printer surveillance

11. 11
In collaboration with Stephanie Moyerman
Video
https://www.youtube.com/watch?v=QA6kCaXufg0

12. Blinking LEDs
12

13. Lighting a LED
13
Resistor (330Ω): limits current to the
LED
Switch turns on and off the LED
 LED is on when the switch is closed
 LED is off when the switch is open
This circuit probably looks very
familiar if you’ve used Arduino

14. Using transistors as
switches
14
SparkFun recommends using
transistors to light LEDs
Transistor basics:
 A small amount of current is needed to
use a transistor as a switch
 A transistor is controlled by the voltage
at the base pin
Reference: SparkFun
≈

15. Wiring up the circuit
15

16. Interfacing with the GPIOs using MRAA
16
MRAA is a low level skeleton library for
IO communication on Edison (and
Galileo and other platforms)
It is a C / C++ library that also provides
Python and JavaScript bindings

17. import mraa
# Other libraries we need
import collections
import time
# Map GPIO block pins to MRAA pin numbers
pins = collections.OrderedDict()
pins["GP44"] = 31
pins["GP45"] = 45
pins["GP46"] = 32
pins["GP47"] = 46
# Initialize LED controls
leds = collections.OrderedDict()
leds["G"] = mraa.Gpio(pins["GP44"])
leds["R"] = mraa.Gpio(pins["GP45"])
leds["W"] = mraa.Gpio(pins["GP46"])
leds["Y"] = mraa.Gpio(pins["GP47"])
17
Python

18. # Set direction of LED controls to out
for color in leds:
leds[color].dir(mraa.DIR_OUT)
LED_ON = 1
LED_OFF = 0
while True:
# Turn all LEDs on for 1 second
for color in leds:
leds[color].write(LED_ON)
time.sleep(1)
# Turn all LEDS off for 1 second
for color in leds:
leds[color].write(LED_OFF)
time.sleep(1)
18
Python

19. 19
Video
https://www.youtube.com/watch?v=YM24yuBLMrU

20. Speech-activated LEDs
20

21. Speech recognition tools
21
Pocketsphinx: a lightweight version of
CMU Sphinx for natural language
processing
Sphinx Knowledge Base Tool: builds a
set of lexical and language modeling
files for the Pocketsphinx decoder
Corpus
• Set of words / phrases that the
decoder should recognize
• Ex: GREEN, RED, WHITE,
YELLOW, ALL
Modeling files
• Dictionary: Provides a
pronunciation for every word-
like string in a corpus
• Language model: Probabilistic
model that estimates the
likelihood of words / phrases

22. 22
High-level overview of speech-activated LEDs
Record audio
•In 2-sec chunks
•Write .wav file
Decode speech with
Pocketsphinx
•Use the dictionary and language
model files
•Return array of recognized words
Trigger LEDs
•Take appropriate action based on
the recognized words
•Ex: If “RED” in recognized words,
toggle on RED LED

23. GitHub
https://github.com/drejkim/led-speech-edison
23

24. 24
Video
https://www.youtube.com/watch?v=Cq_Jf6MInqE

25. 25

26. Edison resources
26
maker.intel.com
Intel IoT Developer Zone
SparkFun Edison products
Edison projects on hackster.io
Edison projects on instructables.com

27. 27
Visit Intel @ THE HUB
2:00-4:45pm