Describe what is the Soundlly 2nd screen services, whose core technology, Soundlly's aerial acoustic communication.

1. Jaewook Kang PhD,
KIT seminar, 24th March 2017
Aerial Acoustic Communication
via Soundlly Technology
Jaewook Kang, Ph.D
jwkang@soundl.ly

2. Jaewook Kang PhD,
KIT seminar, 24th March 2017
The idea is..
Indoor environment
Smart devices and TVs
2
Hmmm looks
yummy.. I want a
recipe for that!

3. Jaewook Kang PhD,
KIT seminar, 24th March 2017
The idea is..
Indoor environment
Smart devices and TVs
Data transmission via near
ultrasounds
3
Signal ID
embedded in the cook show

4. Jaewook Kang PhD,
KIT seminar, 24th March 2017
The idea is..
Indoor environment
Smart devices and TVs
Data transmission via near
ultrasounds
Back-end server
4
ID 30?
URL address
http://www.recipe.kr/…

5. Jaewook Kang PhD,
KIT seminar, 24th March 2017
The idea is..
5
Indoor environment
Smart devices and TVs
Data transmission via near
ultrasounds
Back-end server
Contents acquired!
URL address
http://www.recipe.kr/…
Recipe
ID 30?

6. Jaewook Kang PhD,
KIT seminar, 24th March 2017
It's better to watch than listen a hundred times!
• https://www.youtube.com/watch?v=P0p8pw7Zxmg
6

7. Jaewook Kang PhD,
KIT seminar, 24th March 2017
Service is working over real-world !!
• Soundlly helps TV/video audiences receive useful
information or rewards on smart devices
7
KT CLiP 과 2nd-Screen 광고 서비스 중 (2017.3~)

8. Jaewook Kang PhD,
KIT seminar, 24th March 2017
Service is working over real-world !!
• Soundlly helps TV/video audiences receive useful
information or rewards on smart devices
8
태광그룹 쇼핑엔T 홈쇼핑 채널
(’15. 11 ~ ’16. 4*)
예매앱을 활용한 롯데시네마 극장광고-
스마트폰 연계 광고 (2016.10 ~)
재방문 시 2,000원
할인쿠폰!

9. Jaewook Kang PhD,
KIT seminar, 24th March 2017
Academic Milestones
2013. 사운들리-서울대 서울시 기술개발 과제 진행
2013. 사운들리-한양대 중기청 산학협력 과제 진행
2015. 사운들리-서울대-한양대 K-ICT 기술개발 진행중
논문명: Near-Ultrasound Communication
Modem Design for TV’s 2nd Screen Services
논문명: Chirp Signal-Based Aerial Acoustic
Communication for Smart Devices

10. Jaewook Kang PhD,
KIT seminar, 24th March 2017
Wanted!!
10
사운들리와 함께 글로벌 시장을 혁신할 팀원을 찾습니다!
근무 조건 등 자세한 공고는 job@soundl.ly에 문의
ANDROID 개발 채용
- 업무: 사운들리 SDK 및 앱 개발 및 유지보수
- Android Service, RPC, NDK, GCM 개발 경력
. REST API 를 이용한 클라이언트 개발 경력
. Git 코드 관리 숙련자
- 우대: 앱 퍼블리싱 및 지속 업데이트 경험
개인 레파지토리(github 등) 지속 운영 경험
신호처리 엔지니어
- 업무: 음파신호 송수신기 연구개발
- Strong DSP background
- Detection and estimation theory
- Machine learning
- Communication theory
- C/ C++ / MATLAB / Python
- 우대: 신호처리/무선통신/음향 박석사학위자
사운들리는 안 들리는 소리를 사용해서 기기간
연결을 하는 UX를 세상에서 가장 간편하게 만드는
기술 기반 B2B 스타트업입니다.

11. Jaewook Kang PhD,
KIT seminar, 24th March 2017
Technology: Aerial Acoustic Communications
• System Model
• Acoustic channels
• Speaker – Air medium – Microphone
• Received signal model
11
Pinot Noir
Order More
Tx: Speaker Indoor Aerial
Acoustic Channel
Rx: Smart dev. Mic.

12. Jaewook Kang PhD,
KIT seminar, 24th March 2017
Indoor Aerial Acoustic Channels
• Delay Spread in Indoor Office Environment
• 40 msec delay spread @ –6 dB threshold
5 m, LOS, open space
5 m, NLOS indoor
12

13. Jaewook Kang PhD,
KIT seminar, 24th March 2017
Related Works
• Academic Researches
• Acoustic OFDM [Matsuoka06]
• Embedding information in 6.4~8 kHz of source audio
• 240 bps in a few meters
• Dhwani [Nandakumar13]
• OFDM w/ BPSK @ 6~7 kHz
• 800 bps @ < 20 cm (substituting NFC)
• Commercial Services
• Naratte Zoosh [Zoosh]
• Ultrasound-transmitting screen in taxis
• Payment system & App
• ShopKick [ShopKick]
• Ultrasound transmitter at the entrance of malls
• Provide coupons and/or discount information
13

14. Jaewook Kang PhD,
KIT seminar, 24th March 2017
Limitation of Related Works
Audible Band
Signal
Specific
Interfaces
Short
Range
Audible band (< 18 kHz)
Requires audio source
Custom transmitters
Flat-frequency devices
Up to a few meters
14

15. Jaewook Kang PhD,
KIT seminar, 24th March 2017
Strengths of Technology
• Advantages of the Solution
Inaudible Signal General Audio Interface
Long Range Low battery consumption
Inaudible high frequency band
Up to 25 m in indoor LOS environment Low-power content recognition
Supports frequency-selective devices
15

16. Jaewook Kang PhD,
KIT seminar, 24th March 2017
Inaudible signal with small pressure
16
1KHZ
15KHZ
19KHZ

17. Jaewook Kang PhD,
KIT seminar, 24th March 2017
Popular PHY Modulation Schemes
17
Frequency Shift Keying (FSK)
- Low complexity
- Robust against fading
- Low data rate
- 예) HAM, FM radio
Phase Shift Keying (PSK)
- High complexity
- Vulnerable to fading
- High data rate
- 예) 802.11, CDMA, LTE

18. Jaewook Kang PhD,
KIT seminar, 24th March 2017
Popular PHY Modulation Schemes
18
Frequency Shift Keying (FSK)
- Low complexity
- Robust against fading
- Low data rate
- 예) HAM, FM radio
Phase Shift Keying (PSK)
- High complexity
- Vulnerable to fading
- High data rate
- 예) 802.11, CDMA, LTE

19. Jaewook Kang PhD,
KIT seminar, 24th March 2017
FSK is working bad in high frequency band
• An example: Binary FSK
• Bit ‘0’ with 19 kHz tone
• Bit ‘1’ with 22 kHz tone
• Significant distortion due to MIC freq. response.
19
Freq.19k 22k
Speaker/
MIC response
22 kHz tones disappear!
Galaxy S3, 5m Line of Sight Galaxy S4, 5m Line of Sight
22kHz tone
Region
19kHz tone
Region
22kHz tone
Region
19kHz tone
Region

20. Jaewook Kang PhD,
KIT seminar, 24th March 2017
• Magnitude response of Smart Device MIC (무향실 tx-rx 거리 1m)
•
• Severe frequency selectivity over 20 kHz
20
Smart device MIC characteristic

21. Jaewook Kang PhD,
KIT seminar, 24th March 2017
We use Linear Frequency Modulation (Chirp) !
• Revisit “Rader pulse coding based technology”
• High received power + good resolution
• But Trade off! btw signal power ßà resolution
• Linear FM (Chirp) tech. enables high power
transmission while maintaining a constant bandwidth.
21
[The Figure is
borrowed from
C. Allen @The
University of Kansas]

22. Jaewook Kang PhD,
KIT seminar, 24th March 2017
We use Linear Frequency Modulation (BOK) !
• Binary Orthogonal Keying (BOK) Chirp
• 1 bit / symbol
𝑠" 𝑡 = cos 2𝜋𝑓" 𝑡 +
𝜇𝑡-
2
𝑠- 𝑡 = cos 2𝜋𝑓- 𝑡 −
𝜇𝑡-
2
Up chirp Down chirp
1 1
22
𝜇 (sweep rate)
22
[Winkley62] M. R. Winkley, “Chirp signals for communications,” in WESCON Convention Record, vol.
14, no. 2, 1962.
[Berni73] A. J. Berni and W. Gregg, “On the utility of chirp modulation for digital signaling,” IEEE Trans.
Commun., vol. 21, no. 6, June 1973.

23. Jaewook Kang PhD,
KIT seminar, 24th March 2017
We use Linear Frequency Modulation (BOK) !
• BOK correlation properties
• Demodulation example
23
Frequency
time
0 1 0 1 0

24. Jaewook Kang PhD,
KIT seminar, 24th March 2017
• Receiver Process
24
Recording
Frame
synchronization
Symbol
demodulation
Preamble Preamble
Acoustic Modem Basic Architecture
Single Frame
• Signal of the same ID is repeatedly played
• Record a signal of frame length by microphone in
smart device to guarantee recording a whole
preamble

25. Jaewook Kang PhD,
KIT seminar, 24th March 2017
• Receiver Process
25
Recording
Frame
synchronization
Symbol
demodulation
• Signal of the same ID is repeatedly played
• Record a signal of frame length by microphone in
smart device to guarantee recording a whole
preamble
Preamble Preamble
Single Frame
Acoustic Modem Basic Architecture

26. Jaewook Kang PhD,
KIT seminar, 24th March 2017
• Receiver Process
26
• Frame synchronization by preamble detection
• Matched filtering (convolution)
Recording
Frame
synchronization
Symbol
demodulation
Preamble
Acoustic Modem Basic Architecture

27. Jaewook Kang PhD,
KIT seminar, 24th March 2017
• Receiver Process
27
Envelope output of preamble and up/down chirp.
Recording
Frame
synchronization
Symbol
demodulation
Preamble
Acoustic Modem Basic Architecture

28. Jaewook Kang PhD,
KIT seminar, 24th March 2017
Performance Evaluation
• Demodulation Example
• Recording size (1.1 sec)
• Frame size (1.1 sec)
• BOK chirp symbols
• Guarantees at least one full preamble per recording
• Decoding process
• Preamble detection à Data demodulation
Preamble
offset
Preamble
Recorded
AudioData
Envelope
2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 1
U U U U D D U D D U D U D D D U Preamble
28

29. Jaewook Kang PhD,
KIT seminar, 24th March 2017
Performance Evaluation
• Long-range Reception Ratio
• Most devices can receive with 97% probability
10 ~ 25 m
Randomly-moving people
29
97%

30. Jaewook Kang PhD,
KIT seminar, 24th March 2017
Future directions
• Faster Response !
• More datarate!
• Smaller false-alarm Rate
• Precise MAC protocal
30

31. Jaewook Kang PhD,
KIT seminar, 24th March 2017
Thank You