Kurento cpmx

681 views
544 views

Published on

Published in: Technology, Education
0 Comments
3 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
681
On SlideShare
0
From Embeds
0
Number of Embeds
12
Actions
Shares
0
Downloads
11
Comments
0
Likes
3
Embeds 0
No embeds

No notes for slide

Kurento cpmx

  1. 1. Real-time Multimedia Stream Processing Developing rich multimedia applications with Kurento The Stream oriented GE: Developing rich multimedia applications with Kurento igracia@naevatec.com   h0p://www.kurento.org   Open APIs for Open Minds h"ps://join.me/kurentoCPMX  
  2. 2. Mul8media  infrastructures  for  the   Future  Internet   Media  is   here   Media  got   there   Media  got   there   Analyze   Transform   Store   Transport   Enrich   Augment   Adapt   Sensors   Context   Events   Media  is   here   2  
  3. 3. Enrich,  augment,  adapt,  analyze,   transform,  store:  what’s  the  problem?     Complexity   3  
  4. 4. Future   Internet   Mul8media   Infrastructure   Simple   Development   APIs   Kurento:  the  equa8on   4  
  5. 5. • Interoperable  media  exchange  (mul8plaUorm/mul8protocol)   • WebRTC,  RTP,  HTTP  (video  tag),  etc.   • Process  media  (Computer  vision,  augmented  reality,  media  indexing,  etc.)   • Media  and  metadata  recording  and  recovery   • Transform  and  adapt  media  (H.264,  H.263,  VP8,  Ogg,  and  others)   • Media  rou8ng  and  mixing   • Etc.   Mul8media  infrastructure   •  REST  API   •  JavaScript  API   •  Java  API   APIs   •  LGPL  2.1   Is  distributed  through  a  flexible  FOSS  license   What’s  Kurento   5  
  6. 6. Kurento  Media  Server  (KMS):  the   nucleus  of  Kurento   •  KMS  is  a  middleware  for  media  streams   – Receives  the  stream   – Process  the  stream   – Issues  the  stream   Send   Receive   Analyze   Augment   Enrich   Transform   Transcode   Record   Process   Replicate   Media   Source   Media   Sink   KMS   6  
  7. 7. The  Media  API:  The  API  for  accessing   KMS  capabili8es   Send   Receive   Analyze   Augment   Enrich   Transform   Transcode   Record   Process   Replicate   Media   Source   Media   Sink   KMS   Java   Media  API   JavaScript   Media  API   REST   API   Applica8ons  define  the  processing   of  streams  geeng  through  KMS   7  
  8. 8. Media  API:  Media  Elements  and  Media   Pipelines   Sink%SRC% Sink% SRC% SRC%Sink% Sink% § Media Element • Provides a specific media functionality › Send/receive media › Process media › Transform media • Exchange media through › Sources › Sinks § Media pipeline • Chain of media elements implementing the desired media logic. • The Media API provides the capability of creating media pipelines by joining media elements of the toolbox Media  Element   Sink   SRC   8  
  9. 9. Media  API:  trivial  example  (Java)   MediaPipeline  mp  =  contentSession.getMediaPipelineFactory().create();     PlayerEndpoint  playerEndpoint  =  mp.newPlayerEndpoint(“file:///myFile.webm”).build();     H2pGetEndpoint  h"pEndpoint  =  mp.newH"pGetEndpoint().terminateOnEOS().build();     playerEndpoint.connect(h"pEndpoint);     h"pEndpoint.getUrl();  //URL  where  the  media  is  made  available   Media  Pipeline  (inside  KMS)   H"pGetEndpoint   Media  from   file  or  URI   HTTP  media   streaming   Sink   SRC   PlayerEndpoint   9  
  10. 10. Is  that  enough?  Think  about  the   WWW  development  model   Process  WWW  request   -­‐  DDBB  access   -­‐  Authen8ca8on   -­‐  XML  processing   -­‐  Etc.   HTTP  request:     I  want  this  resource   HTTP  response:   The  resource   •  Intui8on  behind  the  WWW  model     – Client  asks  what  it  wants   •  GET  /this_resource  HTTP/1.1   – Server  side  APIs  execute  the  associated  processing   You  develop  this  using  …   •  Servlet   •  JSP   •  ASP   •  PHP   •  Ruby  on  Rails   •  CGI   •  Etc.   10  
  11. 11. We  need  an  equivalent  model:  the   role  of  the  Signaling  Plane   •  We  need  “something”  with  the  capability  of  nego8a8ng  what,   when  and  how  media  is  exchanged   •  A  protocol  for  “saying”   –  I  want  “this  media”  …   •  Iden8fica8on  of  the  media  to  exchange   –  File  in  hard-­‐drive,  IP  camera,  user,  etc.   –  in  “this  way”  …   •  Iden8fica8on  of  the  processing  of  media   –  Augmented,  analyzed,  etc.   –  with  “this  format”  …   •  Quality   –  Codec,  screen-­‐size,  frame-­‐rate,  etc.   –  at  “this  moment”   •  Stream  control   –  Play,  stop,  start,  pause,  etc.   11  
  12. 12. Don’t  get  it?  think  about  WWW   development  again  …   Process  WWW  request   -­‐  DDBB  access   -­‐  Authen8ca8on   -­‐  XML  processing   -­‐  Etc.   HTTP  request:     I  want  this  resource   HTTP  response:   The  resource   Process  media  request   -­‐  Media  API   -­‐  DDBB  access   -­‐  Authen8ca8on   -­‐  XML  processing   -­‐  Etc.   Signaling  request:     I  want  this  media   Signaling  response:   The  media  is  here   This  is  how  you     create  WWW     applica8ons     (Servlets,  ASP,  PHP,     Rails,  etc.)   This  is  how  you  create   Kurento  enabled   applica8ons:   Mul8media  RTC  is  just   another  feature  of  your   applica8on   You,  as  a     programmer,   create  this   12  
  13. 13. Abstrac8ng  the  signaling  plane:  The   Content  Handler   KMS   Sink  SRC   Sink   SRC   Sink   SRC  Sink   Media  API   REST  API  (Open  API  protocol)   The  Content  Handler   Equivalent  to  a  Servlet/ASP/PHP  script   Exposes  APIs  for  specifying:   -­‐  when  receiving  “this  request”…   -­‐  …  execute  “this  logic”   Developer  can  use  the  media  API   Code  building  the  media  pipeline  and   execu8ng  the  applica8on  logic  the   developer  wants   13  
  14. 14. Kurento  Applica8on  Server:  The   container  of  Handlers   •  Is  an  extension  of  Java  EE  technologies.   •  Compa8ble  with  all  Java  EE  Servlet   containers   •  Hold  Handlers   –  Your  specific  applica8on  code   •  Receives  signaling  requests:   –  I  want  “this  media”  in  this  way  …   •  Dispatches  request  to  the  appropriate   handler   –  @annota8ons  based  mapping   •  Generate  an  answer  showing  “where   and  how  to  find  the  media”   –  URL  where  media  is  located   –  Nego8ated  SDP   Java  EE  compaJble  container   HTTP     Servlet   SIP   Servlet   Web   services   Kurento     REST  API   Specific  handler  implementa8ons   Media  API   DD.BB.   Kurento  ApplicaJon  Server  (KAS)   Other  java   APIs.   14  
  15. 15. Kurento  Architecture:  pueng  it  all   together   Kurento  Media  Server  (KMS)   Receive   Video   Augmented   Reality   Send   Video   Computer     Vision   Video  Playing   and  Recording   Java  EE  compaJble  container   HTTP     Servlet   SIP   Servlet   Web   services   Kurento     REST  API   Specific  handler  implementa8ons   Signaling  and     WWW  traffic   Media   Media   Signaling  and     WWW  traffic   Media  API   DD.BB.   Kurento  ApplicaJon  Server  (KAS)   Other  java   APIs.   15  
  16. 16. Applica8on  execu8on  flow   Client   Code   Applica8on   Server  (KAS)   Media   Server  (KMS)   I  want  this  media  in  this  way  …   (JSON-­‐RPC  request)   Commands  reques8ng   the  crea8on  of  a  pipeline   What  you  want  is  here  …   (JSON-­‐RPC  answer)   Media     negoJaJon   phase   Media     exchange   phase   1   2   Specific  applica8on   logic  at  the     server-­‐side   (Content  Handler)   Media   pipeline   creaJon   Media  exchange  between  client  and  server   16  
  17. 17. Content  Handler:  trivial  example   //Specify  the  type  of  service  provided  by  this  handler:  Player,  Recorder,  WebRTC,  RTP,  etc.   @H2pPlayerService(path  =  "/player”)    //Mapping  of  handler  specified  in  path   public  class  MyPlayerHandler  extends  H"pPlayerHandler  {       @Override   public  void  onContentRequest(H"pPlayerSession  contentSession)  {    //Thie  client  wants  the  media  this  handler  provides    //Create  the  pipeline  for  providing  the  media   }       @Override   public  void  onContentStarted(H"pPlayerSession  contentSession)  {    //Media  started  flowing,  you  can  execute  addi8onal  ac8ons   }     @Override   Public  void  onSessionTerminated(H"pPlayerSession  contentSenssion){    //Media  exchange  termianted,  you  can  collect  your  resources   }     17  
  18. 18. Let’s  develop  with  Kurento   •  What  you  need   –  A  Kurento  instance   •  You  can  install  your  own  Kurento  instance   •  You  can  launch  a  Kurento  instance  at  the  FI-­‐LAB  (FI-­‐WARE  project)   –  h"p://lab.fi-­‐ware.org     •  Geeng  help   –  FI-­‐WARE  catalogue   •  h"p://catalogue.fi-­‐ware.org/enablers/stream-­‐oriented-­‐kurento   –  Kurento  web  site   •  h"p://www.kurento.org   –  Kurento  mailing  list   •  h"ps://groups.google.com/forum/#!forum/kurento   –  Twi"er   •  @kurentoms    
  19. 19. •  FI-­‐LAB   –  Working  instance  of  FI-­‐WARE  enabling  free  experimenta8on  with   technology   –  h"p://lab.fi-­‐ware.org   •  Crea8ng  a  Kurento  instance  from  an  image   –  Use  latest  version  of  Kurento  images.   •  Crea8ng  a  Kurento  instance  using  recipes   –  Use  Ubuntu  13.10  clear  image   –  Use  latest  version  of  Kurento  recipes   •  Accessing  the  Kurento  demo  at  FI-­‐LAB   –  h"p://kurento.lab.fi-­‐ware.org:8080/fi-­‐lab-­‐demo       19   Kurento  at  the  FI-­‐LAB  
  20. 20. Kurento  Hello  World:  Playing  a  file   with  an  HTML5  client   Media  Pipeline   H"pGetEndpoint   Media  from   file  or  URI   HTTP  media   streaming   Sink   SRC   PlayerEndpoint   Media  API   REST  API  (Open  API  protocol)   Handler  code  I  want     “this  media”   Media  is     “at  this  URL”   20  
  21. 21. Playing  a  file  with  an  HTML5  client:   Handler  code  @H"pPlayerService(path  =  "/player”)   public  class  MyPlayerHandler  extends  H"pPlayerHandler  {     @Override   public  void  onContentRequest(H"pPlayerSession  contentSession)  throws  Excep8on  {   MediaPipeline  mp  =  contentSession.getMediaPipelineFactory().create();   contentSession.releaseOnTerminate(mp);   PlayerEndpoint  playerEndpoint  =  mp.newPlayerEndpoint(   "h2p://media.w3.org/2010/05/sintel/trailer.webm").build();   contentSession.setA"ribute("player",  playerEndpoint);   H2pGetEndpoint  h2pEndpoint  =  mp.newH2pGetEndpoint().terminateOnEOS().build();   playerEndpoint.connect(h2pEndpoint);   contentSession.start(h"pEndpoint);   }     @Override   public  void  onContentStarted(H"pPlayerSession  contentSession)  {   PlayerEndpoint  playerEndpoint  =  (PlayerEndpoint)  contentSession.getA"ribute("player");   playerEndpoint.play();   }   }     Source:   h"ps://github.com/Kurento/kurento-­‐media-­‐framework/blob/develop/kmf-­‐samples/kmf-­‐tutorial/src/main/java/com/kurento/tutorial/ MyPlayerHandler.java     21  
  22. 22. Playing  a  file  with  an  HTML5  client:   client  code  <!DOCTYPE  html>   <html>   <head>   <script  src="./js/kws-­‐content-­‐api.js"></script>   <script>   var  conn;   func8on  start()  {   var  op8ons  =  {    remoteVideoTag  :  "remoteVideo"   };   conn  =  new  kwsContentApi.KwsContentPlayer("./player",  op8ons);   }   func8on  terminate()  {    conn.terminate();    }   </script>   </head>   <body>   <bu"on  onclick="start();">Start</bu"on>   <bu"on  onclick="terminate();">Terminate</bu"on>   <br  />   <video  id="remoteVideo"  autoplay></video>   </body>   </html>   Source:   h"ps://github.com/Kurento/kurento-­‐media-­‐framework/blob/develop/kmf-­‐samples/kmf-­‐tutorial/src/main/ webapp/player.html     22  
  23. 23. Playing  a  file  with  an  HTML5  client:   See  the  example  working   h"ps://www.youtube.com/watch?v=n5BQlhYgGSo     23  
  24. 24. Media  Pipeline   Adding  Computer  Vision   H"pGetEndpoint   Media   from   file  or     URI   HTTP  media   streaming   Sink   SRC   PlayerEndpoint   SRC   Sink   JackVaderFilter   Media  API   REST  API  (Open  API  protocol)   Handler  code  I  want     “this  media”   Media  is   “at  this  URL”   24  
  25. 25. Adding  Computer  Vision:  Handler  code   @H"pPlayerService(path  =  "/playerWithFilter”)   public  class  MyPlayerHandler  extends  H"pPlayerHandler  {     @Override   public  void  onContentRequest(H"pPlayerSession  contentSession)  throws  Excep8on  {   MediaPipeline  mp  =  contentSession.getMediaPipelineFactory().create();   contentSession.releaseOnTerminate(mp);   PlayerEndpoint  playerEndpoint  =  mp.newPlayerEndpoint(   "h"p://media.w3.org/2010/05/sintel/trailer.webm").build();   contentSession.setA"ribute("player",  playerEndpoint);   JackVaderFilter  filter  =  mp.newJackVaderFilter().build();   H"pGetEndpoint  h"pEndpoint  =  mp.newH"pGetEndpoint().terminateOnEOS().build();   filter.connect(h"pEndpoint);   playerEndpoint.connect(filter);   contentSession.start(h"pEndpoint);   }     @Override   public  void  onContentStarted(H"pPlayerSession  contentSession)  {   PlayerEndpoint  playerEndpoint  =  (PlayerEndpoint)  contentSession.getA"ribute("player");   playerEndpoint.play();   }   }     Source:   h"ps://github.com/Kurento/kurento-­‐media-­‐framework/blob/develop/kmf-­‐samples/kmf-­‐tutorial/src/main/java/com/kurento/tutorial/ MyPlayerWithFilter.java     25  
  26. 26. Adding  Computer  Vision:  Client  code   <!DOCTYPE  html>   <html>   <head>   <script  src="./js/kws-­‐content-­‐api.js"></script>   <script>   var  conn;   func8on  start()  {   var  op8ons  =  {    remoteVideoTag  :  "remoteVideo"   };   conn  =  new  kwsContentApi.KwsContentPlayer("./playerWithFilter",  op8ons);   }   func8on  terminate()  {    conn.terminate();    }   </script>   </head>   <body>   <bu"on  onclick="start();">Start</bu"on>   <bu"on  onclick="terminate();">Terminate</bu"on>   <br  />   <video  id="remoteVideo"  autoplay></video>   </body>   </html>   Source:   h"ps://github.com/Kurento/kurento-­‐media-­‐framework/blob/develop/kmf-­‐samples/kmf-­‐tutorial/src/main/ webapp/playerFilter.html     26  
  27. 27. Adding  Computer  Vision:  See  the   example  working   h"ps://www.youtube.com/watch?v=yJAQs23eoXw     27  
  28. 28. WebRTC  loopback   Kurento  Media  Server   WebRTC   Streaming   Media  API   JSON-­‐RPC  API   Handler  code   I  want  “this  media”   (offer  SDP)   Media  is  “here”    (answer  SDP)   Sink  SRC   28   Kurento  ApplicaJon  Server   WebRtcEndpoint   Client  Browser  
  29. 29. WebRTC  loopback:  Handler  code   29 Source:   h"ps://github.com/Kurento/kurento-­‐media-­‐framework/blob/develop/kmf-­‐samples/kmf-­‐tutorial/src/main/java/com/kurento/ tutorial/MyWebRtcLoopback.java     @WebRtcContentService(path  =  "/webRtcLoopback")   public  class  MyWebRtcLoopback  extends  WebRtcContentHandler  {     @Override   public  void  onContentRequest(WebRtcContentSession  contentSession)  throws  Excep8on  {   MediaPipeline  mp  =  contentSession.getMediaPipelineFactory().create();   contentSession.releaseOnTerminate(mp);   WebRtcEndpoint  webRtcEndpoint  =  mp.newWebRtcEndpoint().build();   webRtcEndpoint.connect(webRtcEndpoint);   contentSession.start(webRtcEndpoint);   }   }     29  
  30. 30. WebRTC  loopback:  client  code   30 <!DOCTYPE  html>   <html>   <head>   <script  src="./js/kws-­‐content-­‐api.js"></script>   <script   src="h"ps://webrtc.googlecode.com/svn/trunk/samples/js/base/adapter.js"></script>   <script>   var  conn;   func8on  start()  {   var  op8ons  =  {   localVideoTag  :  "localVideo",   remoteVideoTag  :  "remoteVideo"   };   conn  =  new  kwsContentApi.KwsWebRtcContent("./webRtcLoopback",  op8ons);   }   func8on  terminate()  {   conn.terminate();   }   </script>   </head>   <body>   <bu"on  id="start"  onclick="start();">Start</bu"on>   <bu"on  id="terminate"  onclick="terminate();">Terminate</bu"on>   <br  />   <video  id="localVideo"  autoplay></video>   <video  id="remoteVideo"  autoplay></video>   </body>   </html>   Source:   h"ps://github.com/Kurento/kurento-­‐media-­‐framework/blob/develop/kmf-­‐samples/kmf-­‐tutorial/src/main/ webapp/webrtcLoopback.html     30  
  31. 31. WebRTC  loopback:  see  the  example   working   h"ps://www.youtube.com/watch?v=HaVqO06uuNA     31  
  32. 32. WebRTC  recording   Media  Pipeline   WebRTC   Streaming   Media  API   REST  API  (Open  API  protocol)   Handler  code  I  want     “this  media  (SDP)”   Media  is   “at  here  (SDP)”   Sink  SRC   Sink   Media   to   file  or     URI   32  
  33. 33. WebRTC  recorder:  Handler  code   33 Source   h"ps://github.com/Kurento/kurento-­‐media-­‐framework/blob/develop/kmf-­‐samples/kmf-­‐tutorial/src/main/java/com/kurento/ tutorial/MyWebRtcRecorder.java     Source  of  handler  playing  the  recorded  video:   h"ps://github.com/Kurento/kurento-­‐media-­‐framework/blob/develop/kmf-­‐samples/kmf-­‐tutorial/src/main/java/com/kurento/ tutorial/MyPlayerRecording.java     @WebRtcContentService(path  =  "/webRtcRecorder")   public  class  MyWebRtcRecorder  extends  WebRtcContentHandler  {     @Override   public  void  onContentRequest(WebRtcContentSession  contentSession)  throws  Excep8on  {   MediaPipeline  mp  =  contentSession.getMediaPipelineFactory().create();   contentSession.releaseOnTerminate(mp);   WebRtcEndpoint  webRtcEndpoint  =  mp.newWebRtcEndpoint().build();   RecorderEndpoint  recorderEndpoint  =  mp.newRecorderEndpoint("file:///tmp/recording").build();   contentSession.setA"ribute("recorder",  recorderEndpoint);   webRtcEndpoint.connect(webRtcEndpoint);   webRtcEndpoint.connect(recorderEndpoint);   contentSession.start(webRtcEndpoint);   }     @Override   public  void  onContentStarted(WebRtcContentSession  contentSession)  {   RecorderEndpoint  recorderEndpoint  =  (RecorderEndpoint)  contentSession.getA"ribute("recorder");   recorderEndpoint.record();   }   }   33  
  34. 34. WebRTC  recording:  client  code   34 <!DOCTYPE  html>   <html>   <head>   <script  src="./js/kws-­‐content-­‐api.js"></script>   <script   src="h"ps://webrtc.googlecode.com/svn/trunk/samples/js/base/adapter.js"></script>   <script>   var  conn;   func8on  start()  {   var  op8ons  =  {   localVideoTag  :  "localVideo",   remoteVideoTag  :  "remoteVideo"   };   conn  =  new  kwsContentApi.KwsWebRtcContent("./webRtcRecorder",  op8ons);   }   func8on  terminate()  {   conn.terminate();   }   </script>   </head>   <body>   <bu"on  id="start"  onclick="start();">Start</bu"on>   <bu"on  id="terminate"  onclick="terminate();">Terminate</bu"on>   <a  href="./playerRecording">Play  recorded  content</a>   <br  />   <video  id="localVideo"  autoplay></video>   <video  id="remoteVideo"  autoplay></video>   </body>   </html>   Source:   h"ps://github.com/Kurento/kurento-­‐media-­‐framework/blob/develop/kmf-­‐samples/kmf-­‐tutorial/src/main/webapp/ webrtcRecorder.html     34  
  35. 35. WebRTC  recording:  see  the  example   working   h"ps://www.youtube.com/watch?v=ogN81PGkMuE     35  
  36. 36. WebRTC  one-­‐to-­‐many  (Handler   omi"ed  for  simplicity)   Media  Pipeline   Sink  SRC   SRC  Sink  SRC  Sink  SRC  Sink   36  
  37. 37. WebRTC  one-­‐to-­‐many:  Handler  code   Source:   h"ps://github.com/Kurento/kurento-­‐media-­‐framework/blob/develop/kmf-­‐samples/kmf-­‐tutorial/src/main/java/com/kurento/tutorial/ MyWebRtcOneToMany.java     @WebRtcContentService(path  =  "/webRtcOneToMany")   public  class  MyWebRtcOneToMany  extends  WebRtcContentHandler  {        private  WebRtcEndpoint  firstWebRtcEndpoint;    private  String  sessionId;      @Override    public  synchronized  void  onContentRequest(WebRtcContentSession  contentSession)  throws  Excep8on  {   if  (firstWebRtcEndpoint  ==  null)  {   MediaPipeline  mp  =  contentSession.getMediaPipelineFactory().create();   contentSession.releaseOnTerminate(mp);   firstWebRtcEndpoint  =  mp.newWebRtcEndpoint().build();   sessionId  =  contentSession.getSessionId();   contentSession.releaseOnTerminate(firstWebRtcEndpoint);   firstWebRtcEndpoint.connect(firstWebRtcEndpoint);  //Loopback   contentSession.start(firstWebRtcEndpoint);   }  else  {   MediaPipeline  mp  =  firstWebRtcEndpoint.getMediaPipeline();   WebRtcEndpoint  newWebRtcEndpoint  =  mp.newWebRtcEndpoint().build();   contentSession.releaseOnTerminate(newWebRtcEndpoint);   newWebRtcEndpoint.connect(firstWebRtcEndpoint);   firstWebRtcEndpoint.connect(newWebRtcEndpoint);  //Latest  client  gives  media  to  the  master  endpoint   contentSession.start(newWebRtcEndpoint);   }    }   }   37  
  38. 38. WebRTC  one-­‐to-­‐many:  client  code   38 <!DOCTYPE  html>   <html>   <head>   <script  src="./js/kws-­‐content-­‐api.js"></script>   <script   src="h"ps://webrtc.googlecode.com/svn/trunk/samples/js/base/adapter.js"></script>   <script>   var  conn;   func8on  start()  {   var  op8ons  =  {   localVideoTag  :  "localVideo",   remoteVideoTag  :  "remoteVideo"   };   conn  =  new  kwsContentApi.KwsWebRtcContent("./webRtcOneToMany",  op8ons);   }   func8on  terminate()  {   conn.terminate();   }   </script>   </head>   <body>   <bu"on  id="start"  onclick="start();">Start</bu"on>   <bu"on  id="terminate"  onclick="terminate();">Terminate</bu"on>   <br  />   <video  id="localVideo"  autoplay></video>   <video  id="remoteVideo"  autoplay></video>   </body>   </html>   Source:   h"ps://github.com/Kurento/kurento-­‐media-­‐framework/blob/develop/kmf-­‐samples/kmf-­‐tutorial/src/main/webapp/ webrtcOneToMany.html     38  
  39. 39. WebRTC  one-­‐to-­‐many:  see  the   example  working     h"ps://www.youtube.com/watch?v=TBkrl3fmHWI     39  
  40. 40. WebRTC  game  (Handler  omi"ed  for   simplicity)   Media  Pipeline   Sink  SRC   Sink   SRC   Sink   SRC   40  
  41. 41. WebRTC  game:  Handler  code   Source:   h"ps://github.com/Kurento/kurento-­‐media-­‐framework/blob/develop/kmf-­‐samples/kmf-­‐tutorial/src/main/java/com/kurento/tutorial/ MyWebRtcWithFilters.java     @WebRtcContentService(path  =  "/webRtcWithFilters")   public  class  MyWebRtcWithFilters  extends  WebRtcContentHandler  {     @Override   public  void  onContentRequest(WebRtcContentSession  contentSession)throws  Excep8on  {   final  MediaPipeline  mp  =  contentSession.getMediaPipelineFactory().create();   contentSession.releaseOnTerminate(mp);   final  WebRtcEndpoint  webRtcEndpoint  =  mp.newWebRtcEndpoint().build();   final  PointerDetectorFilter  pointerDetectorFilter  =  mp.newPointerDetectorFilter().build();   final  FaceOverlayFilter  faceOverlayFilter  =  mp.newFaceOverlayFilter().build();   PointerDetectorWindowMediaParam  start  =  new  PointerDetectorWindowMediaParam("start",  100,  100,  280,  380);   start.setImage("h"p://ci.kurento.com/imgs/start.png");   pointerDetectorFilter.addWindow(start);   pointerDetectorFilter.addWindowInListener(new  MediaEventListener<WindowInEvent>()  {   public  void  onEvent(WindowInEvent  event)  {   faceOverlayFilter.setOverlayedImage(   "h"p://ci.kurento.com/imgs/mario-­‐wings.png",   -­‐0.35F,  -­‐1.2F,  1.6F,  1.6F);   }   });     webRtcEndpoint.connect(pointerDetectorFilter);   pointerDetectorFilter.connect(faceOverlayFilter);   faceOverlayFilter.connect(webRtcEndpoint);   contentSession.start(webRtcEndpoint);   }     }   41  
  42. 42. WebRTC  game:  client  code   42 <!DOCTYPE  html>   <html>   <head>   <script  src="./js/kws-­‐content-­‐api.js"></script>   <script   src="h"ps://webrtc.googlecode.com/svn/trunk/samples/js/base/adapter.js"></script>   <script>   var  conn;   func8on  start()  {   var  op8ons  =  {   remoteVideoTag  :  "remoteVideo"   };   conn  =  new  kwsContentApi.KwsWebRtcContent("./webRtcWithFilters",  op8ons);   }   func8on  terminate()  {   conn.terminate();   }   </script>   </head>   <body>   <bu"on  id="start"  onclick="start();">Start</bu"on>   <bu"on  id="terminate"  onclick="terminate();">Terminate</bu"on>   <br  />   <video  id="remoteVideo"  autoplay></video>   </body>   </html>   Source:   h"ps://github.com/Kurento/kurento-­‐media-­‐framework/blob/develop/kmf-­‐samples/kmf-­‐tutorial/src/main/webapp/ webrtcFilters.html     42  
  43. 43. WebRTC  game:  see  the  example   working   h"ps://www.youtube.com/watch?v=5eJRnwKxgbY     43  
  44. 44. Complex  applica8on   44   Result:  h"ps://www.youtube.com/watch?v=PYCw9-­‐4oWWA       Sink  SRC   Sink   Sink   SRC   Sink   SRC   Sink   SRC   Sink   SRC   WebRtcEndpoint   MirrorFilter   PointerDetectorFilter   ChromaFilter   FaceOverlayFilter   Media  Pipeline   •  Beyond  video-­‐conferencing:  receiving  events  from  the  media  stream   –  Immersive  and  personalized  adver8sing   –  Games  and  entertainment   Events  (bu"ons)   Game  or  adver8sement  logic   Change  face  Change  background  
  45. 45. Collabora8ons  welcome   h"p://www.github.com/kurento     Thank  you  very  much  for  your  a"en8on           Sugges8ons,  comments  and  complains:   igracia@naevatec.com   45  

×