Presentation IS-4081, Rabbit: Reinventing Video Chat, by Philippe Clavel at the AMD Developer Summit (APU13) November 11-13, 2013.

1. REINVENTING VIDEO CHAT!

2. ! Rabbit
is
a
new
way
to
share
live
experiences
with
anyone,
anywhere
! The
core
product
is
a
plaMorm
that
enables
unlimited
video
chat
and
content
sharing
! Backed
by
Google
Ventures,
CrunchFund
and
Michael
Birch
! Launched
Private
Beta
in
February
on
Mac,
extending
to
Windows
and
Web
in
Q1
2014
2
|
RABBIT
-­‐
REINVENTING
VIDEO
CHAT
|
NOVEMBER
19,
2013
|
CONFIDENTIAL

3. DEMO
! Rabbit
is
currently
in
Beta
and
available
on
Mac
OS
X
! We
are
working
closely
with
AMD
on
a
Windows
version
! We
expect
a
huge
performance
gain…
Let’s see a demo…!
3
|
RABBIT
-­‐
REINVENTING
VIDEO
CHAT
|
NOVEMBER
19,
2013
|
CONFIDENTIAL

4. WHAT
IS
RABBIT
?
Video chat with an unlimited number of people!
4
|
RABBIT
-­‐
REINVENTING
VIDEO
CHAT
|
NOVEMBER
19,
2013
|
CONFIDENTIAL

5. WHAT
IS
RABBIT
?
Unique video chat experience!
5
|
RABBIT
-­‐
REINVENTING
VIDEO
CHAT
|
NOVEMBER
19,
2013
|
CONFIDENTIAL

6. WHAT
IS
RABBIT
?
Share anything with everyone in a room!
6
|
RABBIT
-­‐
REINVENTING
VIDEO
CHAT
|
NOVEMBER
19,
2013
|
CONFIDENTIAL

7. WHAT
IS
RABBIT
?
Watch together!
7
|
RABBIT
-­‐
REINVENTING
VIDEO
CHAT
|
NOVEMBER
19,
2013
|
CONFIDENTIAL

8. CHALLENGES?
! In
short
Rabbit
allows:
‒ Unlimited
number
of
people
in
video
chat
while
watching
content
together
‒ Share
Anything
you
have
access
to
on
your
computer
‒ Including
Audio…
‒ All
that
with
a
unique
UI
and
UX…
! So
some
tech
challenges:
‒ Stream
Audio/Video
of
30+
streams
of
LIVE
content
‒ Render
30+
streams
with
Audio
Video
Synchronizadon
‒ Guarantee
nice
and
dynamic
UI
at
60fps+
‒ Screencast
any
App
running
on
your
computer
‒ Audio
Capture,
Render
and
Processing….
‒ Allow
unlimited
number
of
people
to
join
a
room
8
|
RABBIT
-­‐
REINVENTING
VIDEO
CHAT
|
NOVEMBER
19,
2013
|
CONFIDENTIAL

9. A
LOT
OF
STREAMS…
9
|
RABBIT
-­‐
REINVENTING
VIDEO
CHAT
|
NOVEMBER
19,
2013
|
CONFIDENTIAL

10. A
LOT
OF
STREAMS…
10
|
RABBIT
-­‐
REINVENTING
VIDEO
CHAT
|
NOVEMBER
19,
2013
|
CONFIDENTIAL

11. STREAM
A
LOT
OF
STREAMS
–
HOW?
! Star5ng
Point:
‒ Use
Network
Friendly
Codecs:
vp8,
Speex
‒ Use
Standards:
RTSP,
RTP,
RTCP
‒ Use
open
source
library
for
network
(Live555)
‒ Target
High
Bandwidth
Environment
! Issue:
‒ Codec
makers:
‒ Vp8
didn’t
output
video
quality
good
enough
for
real-­‐dme
encoding
when
we
started
‒ Opus
beker
than
Speex
for
audio
‒ Standards
are
great,
but….
‒ RTP/RTCP
has
port
rules
that
are
really
painful
(2
ports
per
stream)
‒ RTSP:
command
structure
is
heavily
geared
toward
playing
movies
‒ Live555
is
great
for
networking
and
streaming
but:
‒ Single
Threaded
only
‒ Does
not
scale
past
100
streams
on
server
‒ Some
slow
network
code
(read
byte
by
byte,
etc..)
‒ No
real
documentadon…
11
|
RABBIT
-­‐
REINVENTING
VIDEO
CHAT
|
NOVEMBER
19,
2013
|
CONFIDENTIAL

12. STREAM
A
LOT
OF
STREAMS
–
HOW!
‒ Custom
RTSP/RTP/RTCP
stack:
‒ RTP
uses
one
port
for
all
streams
‒ RTSP
stack
is
just
publish,
play
and
stop…
‒ RTCP
implemented
pardally
for
our
needs
(live
streaming)
‒ Custom
UDP
Framing/Deframing
‒ Error
Cancelladon
on
an
as-­‐needed
basis
‒ Streaming
Server
(Hop)
developed
from
scratch
‒ Support
high
number
of
streams,
5k+
clients
simultaneously
‒ Based
on
RTP
‒ Deals
with
hole
punching
and
NATs
‒ Only
one
port
used
for
streaming
12
|
RABBIT
-­‐
REINVENTING
VIDEO
CHAT
|
NOVEMBER
19,
2013
|
CONFIDENTIAL

13. STREAM
A
LOT
OF
STREAMS
–
KEY
BENEFITS
WITH
AMD!
! While
our
network
layer
does
not
change,
our
encoding
and
decoding
pipeline
gets
great
benefits
from
AMD
hardware.
! Encoding:
VCE
‒ Designed
for
low
power
and
low
latency
encoding
(Ideal
for
real-­‐dme
encoding)
‒ We
are
working
closely
with
AMD
to
use
it
for
all
our
encoding
needs
(5
different
streams
in
real
dme)
‒ We
are
able
to
offload
all
our
encoding
from
CPU
(easy
interop
with
OpenCL)…
! Decoding:
UVD
‒ Using
ffmpeg
for
automadc
hardware
decoding
using
UVD
via
DXVA
(simple
API
call)
‒ When
we
do
computadon
(OpenCL/CV),
we
benefit
from
HSA
SVM
transparently
(less
memcopy)…
‒ UVD
is
used
to
decode
most
of
our
incoming
streams.
VCE and UVD are drastically lowering Rabbit’s CPU usage!
13
|
RABBIT
-­‐
REINVENTING
VIDEO
CHAT
|
NOVEMBER
19,
2013
|
CONFIDENTIAL

14. RENDER
A
LOT
OF
STREAMS
–
HOW?
! Where
we
started:
‒ Let’s
decode
using
ffmpeg
‒ Just
push
the
data
to
the
Cocoa
NSView
‒ Just
push
Audio
to
the
speakers
! Issue:
‒ Decoding
is
slow
and
falls
behind…
‒ Simple
UIview
consumes
too
much
CPU
‒ Audio
is
out
of
synch
and
can
be
choppy…
14
|
RABBIT
-­‐
REINVENTING
VIDEO
CHAT
|
NOVEMBER
19,
2013
|
CONFIDENTIAL

15. RENDER
A
LOT
OF
STREAMS
–
HOW
!
! Full
control
of
UI
render
pipeline:
Go
OpenGL…
‒ A
lot
of
video
data
pushed
(15fps
or
30fps)
‒ Minimize
memcpy
‒ Use
shaders
for
render
opdmizadon
‒ Be
smart
and
lean
on
the
endre
pipeline…
! Full
control
of
Audio
Pipeline
‒ Minimize
memcpy
‒ Use
Low
level
Audio
render
(Core
Audio)
! A/V
Synch
‒ Buffer
both
streams
on
the
rendering
end
‒ Synchronized
pushes
to
render
‒ Cover
for
packet
loss
at
network
layer
15
|
RABBIT
-­‐
REINVENTING
VIDEO
CHAT
|
NOVEMBER
19,
2013
|
CONFIDENTIAL

16. RENDER
A
LOT
OF
STREAMS
–
NOW
WITH
AMD!
Capture
Network
In
To
GPU
To
GPU
OpenCV
filters
Decode
16
|
RABBIT
-­‐
REINVENTING
VIDEO
CHAT
|
NOVEMBER
19,
2013
|
CONFIDENTIAL
Face
Detect
OpenCV
filters
Color
Scale
Crop
Face
Detect
Encode
Color
Scale
Crop
To
CPU
Network
Out
Render

17. RENDER
A
LOT
OF
STREAMS
–
NOW
WITH
AMD!
! HSA:
‒ HSA
seamlessly
shares
memory
directly
with
the
graphic
card…
fewer
copies
are
very
important
for
us
;)
‒ By
using
opdmized
Face
Detecdon,
we
plan
to
drasdcally
enhance
the
user
experience
(people
always
in
focus
in
all
bubble,
etc…)
! OpenCL/OpenCV
:
‒ Scaling
/
pan
and
crop
offloaded
‒ Adaptadve
bilateral
filter
(noise
reducdon)
‒ CLAHE
filter
(normalize
light)
‒ Scale
and
Crop
‒ Audio
focus
calculadon
17
|
RABBIT
-­‐
REINVENTING
VIDEO
CHAT
|
NOVEMBER
19,
2013
|
CONFIDENTIAL

18. RENDER
60
FPS
-­‐
HOW!
! Issue:
‒ Graphics
look
choppy
in
most
Video
Chat
Apps
‒ Priority
is
on
Video
not
Audio
‒ Video
is
CPU
and
GPU
intensive
‒ There
must
be
a
beker
way
;)
! Rabbit
=
Full
game
engine:
‒ Game
loop
‒ Low
level
rendering
‒ Nice
threading
to
handle
UI
separated
from
processing
‒ Minimize
memcpy
and
memory
allocadon
‒ And
lot
of
fun
opdmizadon
on
AMD
too;)
Focus on UI/UX gives a very different feel!
18
|
RABBIT
-­‐
REINVENTING
VIDEO
CHAT
|
NOVEMBER
19,
2013
|
CONFIDENTIAL

19. SCREENCAST
ANY
APP
?
19
|
RABBIT
-­‐
REINVENTING
VIDEO
CHAT
|
NOVEMBER
19,
2013
|
CONFIDENTIAL

20. SCREENCAST
ANY
APP
–
HOW
?
! Star5ng
point:
App
Capture…
Easy!
‒ Let’s
capture
the
images
of
the
App
at
high
fps
‒ Let’s
capture
the
audio
of
the
App
‒ Let’s
broadcast…
! Issue:
‒ Capturing
the
image
of
a
given
window
is
not
as
simple
as
capturing
the
full
desktop
‒ App
only
audio
is
very
complex
to
capture
‒ Nice
quality
broadcast
means
big
NAL
and
potendal
packet
loss
or
delay
20
|
RABBIT
-­‐
REINVENTING
VIDEO
CHAT
|
NOVEMBER
19,
2013
|
CONFIDENTIAL

21. SCREENCAST
ANY
APP
–
HOW
?
! Video
‒ Real-­‐
dme
capture
of
an
App’s
window
‒ Real-­‐dme
encoding
using
x264
on
Mac/VCE
on
AMD
‒ Slicing
used
‒ Added
Forward
Error
Correcdon
(FEC)
layer
to
correct
loss
on
large
NALs
! Audio
‒ No
way
to
capture
App
only
audio
on
Mac
‒ Custom
audio
capture
of
system
audio
on
Mac
‒ Real
dme
encoding
using
Opus
‒ Added
custom
Error
Cancelladon
(EC)
to
support
audio
loss
‒ Custom
render
of
full
Audio
(Rabbit
Ears)
21
|
RABBIT
-­‐
REINVENTING
VIDEO
CHAT
|
NOVEMBER
19,
2013
|
CONFIDENTIAL

22. Q
&
A
Quesdons?
22
|
RABBIT
-­‐
REINVENTING
VIDEO
CHAT
|
NOVEMBER
19,
2013
|
CONFIDENTIAL

23. DISCLAIMER
&
ATTRIBUTION
The
informadon
presented
in
this
document
is
for
informadonal
purposes
only
and
may
contain
technical
inaccuracies,
omissions
and
typographical
errors.
The
informadon
contained
herein
is
subject
to
change
and
may
be
rendered
inaccurate
for
many
reasons,
including
but
not
limited
to
product
and
roadmap
changes,
component
and
motherboard
version
changes,
new
model
and/or
product
releases,
product
differences
between
differing
manufacturers,
sovware
changes,
BIOS
flashes,
firmware
upgrades,
or
the
like.
AMD
assumes
no
obligadon
to
update
or
otherwise
correct
or
revise
this
informadon.
However,
AMD
reserves
the
right
to
revise
this
informadon
and
to
make
changes
from
dme
to
dme
to
the
content
hereof
without
obligadon
of
AMD
to
nodfy
any
person
of
such
revisions
or
changes.
AMD
MAKES
NO
REPRESENTATIONS
OR
WARRANTIES
WITH
RESPECT
TO
THE
CONTENTS
HEREOF
AND
ASSUMES
NO
RESPONSIBILITY
FOR
ANY
INACCURACIES,
ERRORS
OR
OMISSIONS
THAT
MAY
APPEAR
IN
THIS
INFORMATION.
AMD
SPECIFICALLY
DISCLAIMS
ANY
IMPLIED
WARRANTIES
OF
MERCHANTABILITY
OR
FITNESS
FOR
ANY
PARTICULAR
PURPOSE.
IN
NO
EVENT
WILL
AMD
BE
LIABLE
TO
ANY
PERSON
FOR
ANY
DIRECT,
INDIRECT,
SPECIAL
OR
OTHER
CONSEQUENTIAL
DAMAGES
ARISING
FROM
THE
USE
OF
ANY
INFORMATION
CONTAINED
HEREIN,
EVEN
IF
AMD
IS
EXPRESSLY
ADVISED
OF
THE
POSSIBILITY
OF
SUCH
DAMAGES.
ATTRIBUTION
©
2013
Advanced
Micro
Devices,
Inc.
All
rights
reserved.
AMD,
the
AMD
Arrow
logo
and
combinadons
thereof
are
trademarks
of
Advanced
Micro
Devices,
Inc.
in
the
United
States
and/or
other
jurisdicdons.
SPEC
is
a
registered
trademark
of
the
Standard
Performance
Evaluadon
Corporadon
(SPEC).
Other
names
are
for
informadonal
purposes
only
and
may
be
trademarks
of
their
respecdve
owners.
23
|
RABBIT
-­‐
REINVENTING
VIDEO
CHAT
|
NOVEMBER
19,
2013
|
CONFIDENTIAL