0
WebRTC	  Tutorial	  28	  Nov	  2012	     0	  
November 27-29, 2012                                                 South San Francisco Conference CenterWebRTC	  Technic...
WebRTC	  Tutorial	  Topics	  •    What	  is	  WebRTC?	  •    How	  to	  Use	  WebRTC	  –	  Peer	  Connec@on	  •    WebRTC	...
Announcement!	                              •  New	  book	  on	  WebRTC!	                                         –  hQp:/...
What	  is	  WebRTC?	        WebRTC	  Tutorial	  28	  Nov	  2012	     4	  
The	  Browser	  RTC	  Func@on	                                                              •  New	  Browser	  Real-­‐Time...
What’s	  New	    WebRTC	  Tutorial	  28	  Nov	  2012	     6	  
What’s	  New	  Con@nued	           WebRTC	  Tutorial	  28	  Nov	  2012	     7	  
WebRTC	  Support	  of	  Mul@ple	  Media	                                          Microphone	  Audio	                     ...
WebRTC	  Triangle	                                                      Web	  Server	                                     ...
WebRTC	  Trapezoid	                                                                          SIP	  	                      ...
WebRTC	  and	  SIP	                                                                  SIP	                                 ...
WebRTC	  and	  PSTN	                                         Web	  Server	                                Peer	  Connec@on...
How	  to	  Use	  WebRTC	          WebRTC	  Tutorial	  28	  Nov	  2012	     13	  
How	  to	  use	  WebRTC	    Obtain	  Local	                    Add	  more	  media	      Media	               All	  media	 ...
How	  to	  use	  WebRTC	  Obtain	  Local	                  Add	  more	  media	                •  getUserMedia()  Media	   ...
How	  to	  use	  WebRTC	  Obtain	  Local	                  Add	  more	  media	                 •  RTCPeerConnection  Media...
How	  to	  use	  WebRTC	  Obtain	  Local	           Add	  more	  media	              •  addStream()  Media	               ...
How	  to	  use	  WebRTC	  Obtain	  Local	           Add	  more	  media	                        Setup	  Peer	    Media	    ...
WebRTC	  Peer-­‐to-­‐Peer	  Media	               WebRTC	  Tutorial	  28	  Nov	  2012	     19	  
Media	  Flows	  in	  WebRTC	                                             Web	  Server	                                    ...
Media	  without	  WebRTC	                                             Web	  Server	                                       ...
Peer-­‐to-­‐Peer	  Media	  with	  WebRTC	                                                  Web	  Server	                  ...
NAT	  Complicates	  Peer-­‐to-­‐Peer	  Media	                                                    Web	  Server	            ...
Peer-­‐to-­‐Peer	  Media	  Through	  NAT	                                                   Web	  Server	                 ...
P2P	  Media	  Can	  Stay	  Local	  to	  NAT	                                                    Web	  Server	             ...
ICE	  uses	  STUN	  and	  TURN	  Servers	                                                  Web	  Server	     STUN	  Server...
Browser	  Queries	  STUN	  Server	  	                                                Web	  Server	     STUN	  Server	   TU...
TURN	  Server	  Can	  Relay	  Media	                                                 Web	  Server	       STUN	   TURN	  Se...
WebRTC	  Protocols	  and	  IETF	          Standards	               WebRTC	  Tutorial	  28	  Nov	  2012	     29	  
WebRTC	  Protocols	  Applica@on	  Layer	                                                                  ICE	      HTTP	 ...
Data	  Channel	  Protocols	                               Data	  Channel	  Data	                                          ...
A	  Joint	  Standards	  Effort	  •  World	  Wide	  Web	  Consor@um	  (W3C)	  	      –  Standardizing	  APIs	  (Applica@on	 ...
IETF	  RTCWEB	  Documents	            WebRTC	  Tutorial	  28	  Nov	  2012	     33	  
WebRTC	  Security	                                   HTML/CSS/JavaScript	                                                 ...
Codecs	                                                                         	  	  	  	  	  	  	  	  	  	  RFC	  6716	 ...
WebRTC	  W3C	  API	  Overview	              WebRTC	  Tutorial	  28	  Nov	  2012	     36	  
Local	  Media	  Handling	                                                                                                 ...
Local	  Media	  Handling	                                                                                                 ...
Local	  Media	  Handling	                                                                                                 ...
Local	  Media	  Handling	                                                                                                 ...
Local	  Media	  Handling	                                                                                                 ...
Transmitng	  media	  •  Signaling	  channel	     –  Non-­‐standard	     –  Must	  exist	  to	  set	  up	  Peer	  Connec@on...
Peer	  Connec@on	  •  "Links"	  together	  two	  peers	      –  Via	  new RTCPeerConnection()    –  Generates	  Session	  ...
Peer	  Connec@on	  •  Handlers	  for	  ICE	  or	  media	  change	      –  onicecandidate,	  onicechange    –  onaddstream,...
Peer	  Connec@on	  •  New	  Iden@ty	  func@ons	     –  setIdentityProvider(),	        getIdentityAssertion	     –  Used	  ...
Pseudo	  Code	  Walkthrough	             WebRTC	  Tutorial	  28	  Nov	  2012	     46	  
Pseudo	  Code	  •    Looks	  like	  real	  code,	  but	  .	  .	  .	  •    API	  is	  s@ll	  in	  flux,	  so	  .	  .	  .	  •...
Back	  to	  first	  diagram	                                      Microphone	  Audio	                                      ...
Mobile	  browser	  code	  outline	            var signalingChannel = createSignalingChannel          ();                  ...
Mobile	  browser	  produces	  .	  .	  .	                                                                                  ...
func@on	  getMedia()	  navigator.getUserMedia({"audio": true }, function (stream) {  microphone = stream;}, e);           ...
func@on	  getMedia()	      . . .    constraint =      {"video": {"mandatory": {"enumDirection": "front"}}};    navigator.g...
WebRTC Technical Overview and Introduction
WebRTC Technical Overview and Introduction
WebRTC Technical Overview and Introduction
WebRTC Technical Overview and Introduction
WebRTC Technical Overview and Introduction
WebRTC Technical Overview and Introduction
WebRTC Technical Overview and Introduction
WebRTC Technical Overview and Introduction
WebRTC Technical Overview and Introduction
WebRTC Technical Overview and Introduction
WebRTC Technical Overview and Introduction
WebRTC Technical Overview and Introduction
WebRTC Technical Overview and Introduction
WebRTC Technical Overview and Introduction
WebRTC Technical Overview and Introduction
WebRTC Technical Overview and Introduction
Upcoming SlideShare
Loading in...5
×

WebRTC Technical Overview and Introduction

14,487

Published on

Published in: Technology
0 Comments
8 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
14,487
On Slideshare
0
From Embeds
0
Number of Embeds
45
Actions
Shares
0
Downloads
222
Comments
0
Likes
8
Embeds 0
No embeds

No notes for slide

Transcript of "WebRTC Technical Overview and Introduction"

  1. 1. WebRTC  Tutorial  28  Nov  2012   0  
  2. 2. November 27-29, 2012 South San Francisco Conference CenterWebRTC  Technical  Overview  and  Introduc7on  Alan  Johnston   Dan  BurneG  Dis7nguished  Engineer   Director  of  Standards  Avaya   Voxeo  Labs  November  28,  2012   November  28,  2012   1  
  3. 3. WebRTC  Tutorial  Topics  •  What  is  WebRTC?  •  How  to  Use  WebRTC  –  Peer  Connec@on  •  WebRTC  Peer-­‐to-­‐Peer  Media  •  WebRTC  Protocols  and  IETF  Standards  •  WebRTC  W3C  API  Overview  •  Pseudo  Code  Walkthrough  •  What’s  Next?   WebRTC  Tutorial  28  Nov  2012   2  
  4. 4. Announcement!   •  New  book  on  WebRTC!   –  hQp://webrtcbook.com   •  Available  on  Amazon  as   paperback  and  Kindle   eBook   •  Also  iBooks,  B&N  Nook,   etc.   WebRTC  Tutorial  28  Nov  2012   3  
  5. 5. What  is  WebRTC?   WebRTC  Tutorial  28  Nov  2012   4  
  6. 6. The  Browser  RTC  Func@on   •  New  Browser  Real-­‐Time   Web   On-­‐the-­‐wire  protocols   (Signaling)   Communica@on  (RTC)   Server   Func@on  built-­‐in  to   HTTP  or  WebSockets   browsers   JavaScript/HTML/CSS   •  Contains   Other  APIs   RTC  APIs   –  Audio  and  video  codecs   –  Ability  to  nego@ate  peer-­‐ Web   Browser   On-­‐the-­‐wire  protocols   to-­‐peer  connec@ons  Browser   RTC   (Media  or  Data)   Func@on   –  Echo  cancella@on,  packet   Na@ve  OS  Services   loss  concealement   •  In  Chrome  today,  Mozilla   soon,  Internet  Explorer   and  Safari  eventually   WebRTC  Tutorial  28  Nov  2012   5  
  7. 7. What’s  New   WebRTC  Tutorial  28  Nov  2012   6  
  8. 8. What’s  New  Con@nued   WebRTC  Tutorial  28  Nov  2012   7  
  9. 9. WebRTC  Support  of  Mul@ple  Media   Microphone  Audio   Applica@on  Sharing  Video   Front  Camera  Video   Rear  Camera  Video   WebCam  Video   Stereo  Audio   Browser  M   Browser  L         on  Mobile   on  Laptop  •  Mul@ple  sources  of  audio  and  video  are  assumed   and  supported  •  All  RTP  media,  voice  and  video,  and  RTCP  feedback   messages  are  mul@plexed  over  the  same  UDP  port   and  address   WebRTC  Tutorial  28  Nov  2012   8  
  10. 10. WebRTC  Triangle   Web  Server   (Applica@on)   Peer  Connec@on  (Audio,  Video,  and/or  Data)   Browser  M   Browser  L  (Running  HTML5  Applica@on     (Running  HTML5  Applica@on     from  Web  Server)   from  Web  Server)  •  Both  browsers  running  the  same  web  applica@on  from  web   server  •  Peer  Connec@on  established  between  them  with  the  help  of   the  web  server   WebRTC  Tutorial  28  Nov  2012   9  
  11. 11. WebRTC  Trapezoid   SIP     or  Jingle   Web  Server  A   Web  Server  B   (Applica@on  A)   (Applica@on  B)   Peer  Connec@on  (Audio  and/or  Video)   Browser  M   Browser  T  (Running  HTML5  Applica@on     (Running  HTML5  Applica@on     from  Web  Server  A)   from  Web  Server  B)  •  Similar  to  SIP  Trapezoid  •  Web  Servers  communicate  using  SIP  or  Jingle  •  Unclear  how  this  really  works  on  web   WebRTC  Tutorial  28  Nov  2012   10  
  12. 12. WebRTC  and  SIP   SIP   Web  Server     SIP  Server   SIP   Peer  Connec@on  (Audio  and/or  Video)   Browser  M   SIP  Client  •  Peer  Connec@on  appears  as  a  standard  RTP  media  session,  described  by   SDP  •  SIP  Endpoint  must  support  RTCWEB  Media  extensions  (ICE  NAT  Traversal,   Secure  RTP,  etc.)  •  Also  drag-­‐ieh-­‐sipcore-­‐websocket  that  defines  SIP  transport  over   WebSockets   WebRTC  Tutorial  28  Nov  2012   11  
  13. 13. WebRTC  and  PSTN   Web  Server   Peer  Connec@on  (Audio)   Browser  M   PSTN  Gateway   Phone  •  Peer  Connec@on  terminates  on  a  PSTN  Gateway  •  Audio  Only   WebRTC  Tutorial  28  Nov  2012   12  
  14. 14. How  to  Use  WebRTC   WebRTC  Tutorial  28  Nov  2012   13  
  15. 15. How  to  use  WebRTC   Obtain  Local   Add  more  media   Media   All  media  added   Setup  Peer   Connec@on   Peer  Connec@on  established   AQach  Media   AQach  more  media  or  data   or  Data   Either  browser  closes  the  connec@on   Close   Connec@on   WebRTC  Tutorial  28  Nov  2012   14  
  16. 16. How  to  use  WebRTC  Obtain  Local   Add  more  media   •  getUserMedia() Media   –  Audio  and/or  video   All  media  added   –  Constraints   Setup  Peer   Connec@on   –  User  permissions   Peer  Connec@on  established   •  LocalMediaStreamAQach  Media   •  MediaStream AQach  more  media  or  data   or  Data   –  Local  or  derived   Either  browser  closes  the  connec@on   Close   Connec@on   WebRTC  Tutorial  28  Nov  2012   15  
  17. 17. How  to  use  WebRTC  Obtain  Local   Add  more  media   •  RTCPeerConnection Media   –  Direct  media   All  media  added   –  Between  two  peers   Setup  Peer   Connec@on   –  ICE  processing   Peer  Connec@on  established   –  SDP  processing   –  Iden@ty  verifica@on  AQach  Media   AQach  more  media  or  data   or  Data   –  Sta@s@cs  repor@ng   Either  browser  closes  the  connec@on   Close   Connec@on   WebRTC  Tutorial  28  Nov  2012   16  
  18. 18. How  to  use  WebRTC  Obtain  Local   Add  more  media   •  addStream() Media   –  Doesnt  change  media  state!   All  media  added   •  removeStream() Setup  Peer   –  DiQo!   Connec@on   •  createOffer(), Peer  Connec@on  established   createAnswer() •  setLocalDescription(),AQach  Media   AQach  media  or  data   setRemoteDescription() or  Data   •  Applying  SDP  answer  makes   Either  browser  closes  the  connec@on   the  magic  happen   Close   •  createDataChannel() Connec@on   WebRTC  Tutorial  28  Nov  2012   17  
  19. 19. How  to  use  WebRTC  Obtain  Local   Add  more  media   Setup  Peer   Media   Connec@on  All  media  added   Peer  Connec@on  established   AQach  Media   AQach  more  media  or  data   or  Data   Either  browser  closes  the  connec@on   Close   Connec@on   WebRTC  Tutorial  28  Nov  2012   18  
  20. 20. WebRTC  Peer-­‐to-­‐Peer  Media   WebRTC  Tutorial  28  Nov  2012   19  
  21. 21. Media  Flows  in  WebRTC   Web  Server   Internet   Home  WiFi   Router   Router   Browser  D  Browser  M   Browser  T   Coffee  Shop   WiFi  Router   Browser  L   WebRTC  Tutorial  28  Nov  2012   20  
  22. 22. Media  without  WebRTC   Web  Server   Internet   Home  WiFi   Router   Router   Browser  D  Browser  M   Browser  T   Coffee  Shop   WiFi  Router   Browser  L   WebRTC  Tutorial  28  Nov  2012   21  
  23. 23. Peer-­‐to-­‐Peer  Media  with  WebRTC   Web  Server   Internet   Home  WiFi   Router   Router   Browser  D   Browser  M   Browser  T   Coffee  Shop   WiFi  Router   Browser  L   WebRTC  Tutorial  28  Nov  2012   22  
  24. 24. NAT  Complicates  Peer-­‐to-­‐Peer  Media   Web  Server   Most  browsers  are  behind   NATs  on  the  Internet,   which  complicates  the   establishment  of  peer-­‐to-­‐ peer  media  sessions.   Internet   Home  WiFi     Router  with   with  NAT   NAT   Browser  D   Browser  M   Browser  T   Coffee  Shop   WiFi  with   NAT   Browser  L   WebRTC  Tutorial  28  Nov  2012   23  
  25. 25. Peer-­‐to-­‐Peer  Media  Through  NAT   Web  Server   ICE  hole  punching  can   ogen  establish  a  direct   peer-­‐to-­‐peer  session   between  browsers   behind  different  NATs   Internet   Home  WiFi     Router  with   with  NAT   NAT   Browser  D   Browser  M   Browser  T   Coffee  Shop   WiFi  with   NAT   Browser  L   WebRTC  Tutorial  28  Nov  2012   24  
  26. 26. P2P  Media  Can  Stay  Local  to  NAT   Web  Server   If  both  browsers  are   behind  the  same  NAT,  hole   punching  can  ogen   establish  a  connec@on   that  never  leaves  the  NAT.   Internet   Home  WiFi     Router  with   with  NAT   NAT   Browser  D   Browser  M   Browser  T   Coffee  Shop   WiFi  with   NAT   Browser  L   WebRTC  Tutorial  28  Nov  2012   25  
  27. 27. ICE  uses  STUN  and  TURN  Servers   Web  Server   STUN  Server   TURN  Server   198.51.100.9   198.51.100.2   ICE  hole  punching  uses   STUN  and  TURN  servers  in   the  public  Internet  to  help   Internet   with  NAT  traversal.   Home  WiFi     Router  with   with  NAT   NAT   203.0.113.4   Browser  D   Browser  M   192.168.0.5   Coffee  Shop   Browser  T   WiFi  with   NAT   Browser  L   WebRTC  Tutorial  28  Nov  2012   26  
  28. 28. Browser  Queries  STUN  Server     Web  Server   STUN  Server   TURN  Server   198.51.100.9   198.51.100.2   Browser  sends  STUN  test   packet  to  STUN  server  to   learn  its  public  IP  address   Internet   (address  of  the  NAT).   Home  WiFi     Router  with   with  NAT   NAT   203.0.113.4   Browser  D   Browser  M   192.168.0.5   Coffee  Shop   Browser  T   WiFi  with   NAT   Browser  L   WebRTC  Tutorial  28  Nov  2012   27  
  29. 29. TURN  Server  Can  Relay  Media   Web  Server   STUN   TURN  Server  as  a   Server   Media  Relay   In  some  cases,  hole   punching  fails,  and  a  TURN   Media  Relay  on  the  public   Internet   Internet  must  be  used.     Home  WiFi     Router  with   with  NAT   NAT   Browser  D   Browser  M   Browser  T   Coffee  Shop   WiFi  with   NAT   Browser  L   WebRTC  Tutorial  28  Nov  2012   28  
  30. 30. WebRTC  Protocols  and  IETF   Standards   WebRTC  Tutorial  28  Nov  2012   29  
  31. 31. WebRTC  Protocols  Applica@on  Layer   ICE   HTTP   WebSocket   SRTP   SDP   STUN   TURN  Transport  Layer   TLS   DTLS   TCP   SCTP   UDP  Network  Layer   IP   WebRTC  Tutorial  28  Nov  2012   30  
  32. 32. Data  Channel  Protocols   Data  Channel  Data   Provides  conges@on  and  flow  control   SCTP   Provides  security  (confiden@ality)   DTLS   DTLS   Provides  transport  through   UDP   NAT  (ager  ICE  hole  punching)   Internet  •  Data  channel  provides  a  non-­‐media  channel  between   browsers  •  ICE  is  s@ll  used  for  NAT  Traversal  •  Used  in  gaming  and  other  non-­‐communica@on  applica@ons   WebRTC  Tutorial  28  Nov  2012   31  
  33. 33. A  Joint  Standards  Effort  •  World  Wide  Web  Consor@um  (W3C)     –  Standardizing  APIs  (Applica@on  Programming   Interfaces)   –  Most  work  in  WEBRTC  Working  Group   –  Used  by  JavaScript  to  access  RTC  func@on  •  Internet  Engineering  Task  Force  (IETF)   –  Standardizing  protocols  (bits  on  the  wire)   –  Peer  Connec@on  will  use  RTP,  SDP,  and  extensions   –  Some  work  in  RTCWEB  Working  Group   –  Lots  of  related  work  in  MMUSIC,  AVTCORE,  etc.   WebRTC  Tutorial  28  Nov  2012   32  
  34. 34. IETF  RTCWEB  Documents   WebRTC  Tutorial  28  Nov  2012   33  
  35. 35. WebRTC  Security   HTML/CSS/JavaScript   Provides  transport  of  HTML/CSS/JavaScript   HTTP   Provides  security  (confiden@ality   TLS   DTLS   and  authen@ca@on)   TCP   Provides  reliability  and   conges@on  control   Internet  •  Media  is  secured  by  Secure  RTP    •  Control  is  secured  by  HTTPS  (HTTP  over  TLS  over  TCP)  •  Browser  confirms  permissions  for  microphone  and   camera  on  each  session   WebRTC  Tutorial  28  Nov  2012   34  
  36. 36. Codecs                      RFC  6716                  .                •  Mandatory  to  Implement  (MTI)  audio  codecs   are  seQled  (finally!)  •  Video  is  not!   WebRTC  Tutorial  28  Nov  2012   35  
  37. 37. WebRTC  W3C  API  Overview   WebRTC  Tutorial  28  Nov  2012   36  
  38. 38. Local  Media  Handling   Presenta@on  Stream  label=“F8kdls”   Audio   Track  label=“Audio”   Audio  Stream  label=“2dLe3js”   Track  label=“Presenta@on”   Presenta@on  Video   Presenter  Stream  label=“8dFlf”   Video  Stream  label=“eR3l0s”   Track  label=“Audio”   Microphone  Audio   Presenter  Video   Track  label=“Presenter”   Applica@on  Sharing  Video   Demonstra@on  Stream  label=“3dfdf2”   Front  Camera  Video   Video  Stream  label=“js4KMs”   Track  label=“Audio”   Rear  Camera  Video   Demonstra@on  Video   Track  label=“Demonstra@on”  Browser  M   Video  Stream  label=“923fKs”   Sources   Local  Media  Streams   Streams   Tracks   •  Channels   –  Encoded  together   –  Cant  manipulate  individually   WebRTC  Tutorial  28  Nov  2012   37  
  39. 39. Local  Media  Handling   Presenta@on  Stream  label=“F8kdls”   Audio   Track  label=“Audio”   Audio  Stream  label=“2dLe3js”   Track  label=“Presenta@on”   Presenta@on  Video   Presenter  Stream  label=“8dFlf”   Video  Stream  label=“eR3l0s”   Track  label=“Audio”   Microphone  Audio   Presenter  Video   Track  label=“Presenter”   Applica@on  Sharing  Video   Demonstra@on  Stream  label=“3dfdf2”   Front  Camera  Video   Video  Stream  label=“js4KMs”   Track  label=“Audio”   Rear  Camera  Video   Demonstra@on  Video   Track  label=“Demonstra@on”  Browser  M   Video  Stream  label=“923fKs”   Sources   Local  Media  Streams   Streams   Tracks   •  Tracks  (MediaStreamTrack)   –  Exist  only  as  part  of  Streams   –  Ordered  and  op@onally  labeled   WebRTC  Tutorial  28  Nov  2012   38  
  40. 40. Local  Media  Handling   Presenta@on  Stream  label=“F8kdls”   Audio   Track  label=“Audio”   Audio  Stream  label=“2dLe3js”   Track  label=“Presenta@on”   Presenta@on  Video   Presenter  Stream  label=“8dFlf”   Video  Stream  label=“eR3l0s”   Track  label=“Audio”   Microphone  Audio   Presenter  Video   Track  label=“Presenter”   Applica@on  Sharing  Video   Demonstra@on  Stream  label=“3dfdf2”   Front  Camera  Video   Video  Stream  label=“js4KMs”   Track  label=“Audio”   Rear  Camera  Video   Demonstra@on  Video   Track  label=“Demonstra@on”  Browser  M   Video  Stream  label=“923fKs”   Sources   Local  Media  Streams   Streams   Tracks   •  Streams  (MediaStream)   –  All  contained  tracks  are  synchronized   –  Can  be  created,  transmiQed,  etc.   WebRTC  Tutorial  28  Nov  2012   39  
  41. 41. Local  Media  Handling   Presenta@on  Stream  label=“F8kdls”   Audio   Track  label=“Audio”   Audio  Stream  label=“2dLe3js”   Track  label=“Presenta@on”   Presenta@on  Video   Presenter  Stream  label=“8dFlf”   Video  Stream  label=“eR3l0s”   Track  label=“Audio”   Microphone  Audio   Presenter  Video   Track  label=“Presenter”   Applica@on  Sharing  Video   Demonstra@on  Stream  label=“3dfdf2”   Front  Camera  Video   Video  Stream  label=“js4KMs”   Track  label=“Audio”   Rear  Camera  Video   Demonstra@on  Video   Track  label=“Demonstra@on”  Browser  M   Video  Stream  label=“923fKs”   Sources   Local  Media  Streams   Streams   Tracks   •  LocalMediaStream   –  Returned  from  getUserMedia() –  Directly  connected  to  source   –  Permission  check  required  to  obtain   40   WebRTC  Tutorial  28  Nov  2012  
  42. 42. Local  Media  Handling   Presenta@on  Stream  label=“F8kdls”   Audio   Track  label=“Audio”   Audio  Stream  label=“2dLe3js”   Track  label=“Presenta@on”   Presenta@on  Video   Presenter  Stream  label=“8dFlf”   Video  Stream  label=“eR3l0s”   Track  label=“Audio”   Microphone  Audio   Presenter  Video   Track  label=“Presenter”   Applica@on  Sharing  Video   Demonstra@on  Stream  label=“3dfdf2”   Front  Camera  Video   Video  Stream  label=“js4KMs”   Track  label=“Audio”   Rear  Camera  Video   Demonstra@on  Video   Track  label=“Demonstra@on”  Browser  M   Video  Stream  label=“923fKs”   Sources   Local  Media  Streams   Streams   Tracks   •  In  this  example   –  Obtained  4  local  media  streams   –  Created  3  media  streams  from  them   –  Sent  streams  over  Peer  Connec@on   41   WebRTC  Tutorial  28  Nov  2012  
  43. 43. Transmitng  media  •  Signaling  channel   –  Non-­‐standard   –  Must  exist  to  set  up  Peer  Connec@on  •  Peer  Connec@on   –  Links  together  two  peers   –  Add/Remove  Media  Streams   •  addStream(),  removeStream() –  Handlers  for  ICE  or  media  change   –  Data  Channel  support   WebRTC  Tutorial  28  Nov  2012   42  
  44. 44. Peer  Connec@on  •  "Links"  together  two  peers   –  Via  new RTCPeerConnection() –  Generates  Session  Descrip@on  offers/answers   •  createOffer(),  createAnswer() –  From  SDP  answers,  ini@ates  media   •  setLocalDescription(),  setRemoteDescription() –  Offers/answers  MUST  be  relayed  by  applica@on  code!   –  ICE  candidates  can  also  be  relayed  and  added  by  app   •  addIceCandidate() –  Think  of  PC  as  an  applica@on  helper   WebRTC  Tutorial  28  Nov  2012   43  
  45. 45. Peer  Connec@on  •  Handlers  for  ICE  or  media  change   –  onicecandidate,  onicechange –  onaddstream,  onremovestream –  onnegotiationneeded –  A  few  others   WebRTC  Tutorial  28  Nov  2012   44  
  46. 46. Peer  Connec@on  •  New  Iden@ty  func@ons   –  setIdentityProvider(),   getIdentityAssertion   –  Used  to  verify  iden@ty  via  third  party,  e.g.,   Facebook  Connect  •  New  Sta@s@cs  API   –  getStats() –  Obtain  sta@s@cs,  local  and  remote,  on  bytes/ packets  xmiQed,  audio  volume,  etc.   –  May  be  useful  for  conges@on-­‐based  adjustments   WebRTC  Tutorial  28  Nov  2012   45  
  47. 47. Pseudo  Code  Walkthrough   WebRTC  Tutorial  28  Nov  2012   46  
  48. 48. Pseudo  Code  •  Looks  like  real  code,  but  .  .  .  •  API  is  s@ll  in  flux,  so  .  .  .  •  Dont  expect  this  to  work  anywhere  .  .  .  •  Yet   WebRTC  Tutorial  28  Nov  2012   47  
  49. 49. Back  to  first  diagram   Microphone  Audio   Applica@on  Sharing  Video   Front  Camera  Video   Rear  Camera  Video   WebCam  Video   Stereo  Audio   Browser  M   Browser  L         on  Mobile   on  Laptop  •  Mobile  browser  "calls"  laptop  browser  •  Each  sends  media  to  the  other   WebRTC  Tutorial  28  Nov  2012   48  
  50. 50. Mobile  browser  code  outline   var signalingChannel = createSignalingChannel (); var pc; var configuration = {"iceServers":[{"url":"stun:198.51.100.9"}, {"url":"turn:198.51.100.2", "credential":"myPassword"}]}; var microphone, application, front, rear; var presentation, presenter, demonstration; var remote_av, stereo, mono; var display, left, right; Function s(sdp) {{ // stub success callback Function e(error) {} // stub error callback var signalingChannel = createSignalingChannel(); getMedia(); getMedia(); createPC(); attachMedia(); call(); function getMedia() { navigator.getUserMedia({"audio": true }, function (stream) { microphone = stream; }, e); createPC(); // get local video (application sharing) ///// This is outside the scope of this specification. ///// Assume that application has been set to this stream. // constraint = {"video": {"mandatory": {"enumDirection": "front"}}}; navigator.getUserMedia(constraint, function (stream) { front = stream; }, e); attachMedia(); constraint = {"video": {"mandatory": {"enumDirection": "rear"}}}; navigator.getUserMedia(constraint, function (stream) { rear = stream; }, e); } function createPC() { pc = new RTCPeerConnection(configuration); call(); pc.onicecandidate = function (evt) { signalingChannel.send( JSON.stringify({ "candidate": evt.candidate })); }; pc.onaddstream = function (evt) {handleIncomingStream(evt.stream);}; } function attachMedia() { presentation = new MediaStream([microphone.audioTracks.item(0), application.videoTracks.item(0)]); presentation.audioTracks.item(0).label = "Audio"; presentation.videoTracks.item(0).label = "Presentation"; presenter = new MediaStream([microphone.audioTracks.item(0),•  We  will  look  next  at  each  of  these   front.videoTracks.item(0)]); presenter.audioTracks.item(0).label = "Audio"; presenter.videoTracks.item(0).label = "Presenter"; demonstration = new MediaStream([microphone.audioTracks.item(0), rear.videoTracks.item(0)]); demonstration.audioTracks.item(0).label = "Audio"; demonstration.videoTracks.item(0).label = "Demonstration"; pc.addStream(presentation); pc.addStream(presenter); pc.addStream(demonstration); } function call() { pc.createOffer(gotDescription, e);•  .  .  .  except  for  crea@ng  the  signaling   function gotDescription(desc) { pc.setLocalDescription(desc, o, e); signalingChannel.send(JSON.stringify({ "sdp": desc })); } } function handleIncomingStream(s) { if (s.videoTracks.length == 1) { av_stream = s; show_av(av_stream); } else if (s.audioTracks.length == 2) { stereo = s; channel   } else { mono = s; } } function show_av(s) { display.src = URL.createObjectURL(s.videoTracks.item(0)); left.src = URL.createObjectURL(s.audioTracks.item(0)); right.src = URL.createObjectURL(s.audioTracks.item(1)); } signalingChannel.onmessage = function (msg) { var signal = JSON.parse(msg.data); if (signal.sdp) { pc.setRemoteDescription( new RTCSessionDescription(signal.sdp), s, e); } else { pc.addIceCandidate( new RTCIceCandidate(signal.candidate)); } }; WebRTC  Tutorial  28  Nov  2012   49  
  51. 51. Mobile  browser  produces  .  .  .   Presenta@on  Stream  label=“F8kdls”   Audio   Track  label=“Audio”   Audio  Stream  label=“2dLe3js”   Track  label=“Presenta@on”   Presenta@on  Video   Presenter  Stream  label=“8dFlf”   Video  Stream  label=“eR3l0s”   Track  label=“Audio”   Microphone  Audio   Presenter  Video   Track  label=“Presenter”   Applica@on  Sharing  Video   Demonstra@on  Stream  label=“3dfdf2”   Front  Camera  Video   Video  Stream  label=“js4KMs”   Track  label=“Audio”   Rear  Camera  Video   Demonstra@on  Video   Track  label=“Demonstra@on”  Browser  M   Video  Stream  label=“923fKs”   Sources   Local  Media  Streams   Streams   Tracks   •  Four  calls  to  getUserMedia() •  Three  calls  to  new MediaStream() •  App  then  labels  all  tracks  and  sends  them   50   WebRTC  Tutorial  28  Nov  2012  
  52. 52. func@on  getMedia()  navigator.getUserMedia({"audio": true }, function (stream) { microphone = stream;}, e); var pc; var configuration = {"iceServers":[{"url":"stun:198.51.100.9"}, {"url":"turn:198.51.100.2", "credential":"myPassword"}]}; var microphone, application, front, rear; var presentation, presenter, demonstration; var remote_av, stereo, mono; var display, left, right; Function s(sdp) {{ // stub success callback Function e(error) {} // stub error callback// get local video (application sharing) var signalingChannel = createSignalingChannel(); getMedia(); createPC(); attachMedia(); call();///// This is outside the scope of this specification. function getMedia() { navigator.getUserMedia({"audio": true }, function (stream) { microphone = stream; }, e); // get local video (application sharing) ///// This is outside the scope of this specification. ///// Assume that application has been set to this stream. /////// Assume that application has been set to this stream. constraint = {"video": {"mandatory": {"enumDirection": "front"}}}; navigator.getUserMedia(constraint, function (stream) { front = stream; }, e); constraint = {"video": {"mandatory": {"enumDirection": "rear"}}};// navigator.getUserMedia(constraint, function (stream) { rear = stream; }, e); } function createPC() { pc = new RTCPeerConnection(configuration); pc.onicecandidate = function (evt) { signalingChannel.send( JSON.stringify({ "candidate": evt.candidate })); }; pc.onaddstream = function (evt) {handleIncomingStream(evt.stream);}; } function attachMedia() {. . . presentation = new MediaStream([microphone.audioTracks.item(0), application.videoTracks.item(0)]); presentation.audioTracks.item(0).label = "Audio"; presentation.videoTracks.item(0).label = "Presentation"; presenter = new MediaStream([microphone.audioTracks.item(0), front.videoTracks.item(0)]); presenter.audioTracks.item(0).label = "Audio"; •  Get  audio   presenter.videoTracks.item(0).label = "Presenter"; demonstration = new MediaStream([microphone.audioTracks.item(0), rear.videoTracks.item(0)]); demonstration.audioTracks.item(0).label = "Audio"; demonstration.videoTracks.item(0).label = "Demonstration"; pc.addStream(presentation); pc.addStream(presenter); pc.addStream(demonstration); } function call() { pc.createOffer(gotDescription, e); function gotDescription(desc) { pc.setLocalDescription(desc, s, e); •  (Get  window  video  –  out  of  scope)   signalingChannel.send(JSON.stringify({ "sdp": desc })); } } function handleIncomingStream(s) { if (s.videoTracks.length == 1) { av_stream = s; show_av(av_stream); } else if (s.audioTracks.length == 2) { stereo = s; } else { mono = s; } } function show_av(s) { display.src = URL.createObjectURL(s.videoTracks.item(0)); left.src = URL.createObjectURL(s.audioTracks.item(0)); right.src = URL.createObjectURL(s.audioTracks.item(1)); } signalingChannel.onmessage = function (msg) { var signal = JSON.parse(msg.data); if (signal.sdp) { pc.setRemoteDescription( new RTCSessionDescription(signal.sdp), s, e); } else { pc.addIceCandidate( new RTCIceCandidate(signal.candidate)); } }; WebRTC  Tutorial  28  Nov  2012   51  
  53. 53. func@on  getMedia()   . . . constraint = {"video": {"mandatory": {"enumDirection": "front"}}}; navigator.getUserMedia(constraint, function (stream) { var pc; var configuration = {"iceServers":[{"url":"stun:198.51.100.9"}, {"url":"turn:198.51.100.2", front = stream; "credential":"myPassword"}]}; var microphone, application, front, rear; var presentation, presenter, demonstration; var remote_av, stereo, mono; var display, left, right; Function s(sdp) {{ // stub success callback Function e(error) {} // stub error callback }, e); var signalingChannel = createSignalingChannel(); getMedia(); createPC(); attachMedia(); call(); function getMedia() { navigator.getUserMedia({"audio": true }, function (stream) { microphone = stream; }, e); // get local video (application sharing) ///// This is outside the scope of this specification. ///// Assume that application has been set to this stream. // constraint = constraint = {"video": {"mandatory": {"enumDirection": "front"}}}; navigator.getUserMedia(constraint, function (stream) { front = stream; }, e); constraint = {"video": {"mandatory": {"enumDirection": "rear"}}}; navigator.getUserMedia(constraint, function (stream) { rear = stream; {"video": {"mandatory": {"enumDirection": "rear"}}}; }, e); } function createPC() { pc = new RTCPeerConnection(configuration); pc.onicecandidate = function (evt) { signalingChannel.send( JSON.stringify({ "candidate": evt.candidate })); navigator.getUserMedia(constraint, function (stream) { }; pc.onaddstream = function (evt) {handleIncomingStream(evt.stream);}; } function attachMedia() { presentation = new MediaStream([microphone.audioTracks.item(0), rear = stream; application.videoTracks.item(0)]); presentation.audioTracks.item(0).label = "Audio"; presentation.videoTracks.item(0).label = "Presentation"; presenter = new MediaStream([microphone.audioTracks.item(0), front.videoTracks.item(0)]); presenter.audioTracks.item(0).label = "Audio"; presenter.videoTracks.item(0).label = "Presenter"; demonstration = new MediaStream([microphone.audioTracks.item(0), }, e); rear.videoTracks.item(0)]); demonstration.audioTracks.item(0).label = "Audio"; demonstration.videoTracks.item(0).label = "Demonstration"; pc.addStream(presentation); pc.addStream(presenter); pc.addStream(demonstration); } function call() { pc.createOffer(gotDescription, s, e);•  Get  front-­‐facing  camera   function gotDescription(desc) { pc.setLocalDescription(desc, s, e); signalingChannel.send(JSON.stringify({ "sdp": desc })); } } function handleIncomingStream(s) { if (s.videoTracks.length == 1) { av_stream = s; show_av(av_stream); } else if (s.audioTracks.length == 2) { stereo = s; } else { mono = s; }•  Get  rear-­‐facing  camera   } function show_av(s) { display.src = URL.createObjectURL(s.videoTracks.item(0)); left.src = URL.createObjectURL(s.audioTracks.item(0)); right.src = URL.createObjectURL(s.audioTracks.item(1)); } signalingChannel.onmessage = function (msg) { var signal = JSON.parse(msg.data); if (signal.sdp) { pc.setRemoteDescription( new RTCSessionDescription(signal.sdp), s, e); } else { pc.addIceCandidate( new RTCIceCandidate(signal.candidate)); } }; WebRTC  Tutorial  28  Nov  2012   52  
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×