Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Kurento: a media server architecture
and API
Live video technology

Luis Lopez
lulop@kurento.org
Who am I?
Associate Prof.
• Computer Science
• Multimedia protocols and systems

15 years of experience in multimedia appl...
What’s Kurento?
Kurento
• Esperanto translation of “stream”

Esperanto philosophy
• Openness
• Simplicity
• Interoperabili...
Kurento’s history
2004

March 2013

• Different research
efforts from private
and public
organizations in
the area of
mult...
Why WebRTC?
•

WebRTC is a movement
– … by Tsahi Levent-Levi

•

Are RTCs new?
– Skype, H.323, Facetime, WebEx

•

Why Web...
Why Kurento?
WWW RTC developer experience
Before WebRTC
Developing
the client side

After WebRTC

Begin

Next natural step...
Kurento media framework
components
•Low level media
capabilities

•Signaling and control
capabilities
•SIP, HTTP, RESFul, ...
Kurento Application Server: extending
the WWW development model
Signaling request:
I want this media

HTTP request:
I want...
Media API: media elements and media
pipelines
 Media Element
• Provides a specific media functionality

•

Send/receive m...
Application
architecture

Other systems
and databases

Signaling and
WWW traffic

Media

Media

Signaling and
WWW traffic
...
Possible use cases: just integrate with
Java EE and GStreamer
• Verticals
– E-Health
• Kurento + HAPI (http://hl7api.sourc...
Application example: requirements

Kurento Media Server Infrastructure

: a media server architecture and API – http://www...
Application example: code
@WebRtcMediaService(name = "MyWebRtcService", path = "/pathToService")
public class MyWebRtcServ...
Application example: result

: a media server architecture and API – http://www.kurento.org

14
Media element toolbox
Transport

Repository

Group communications

• WebRtcEndpoint
• RtpEndpoint
• HttpEndpoint

• Player...
Media elements: WebRtcEndpoint
• Full implementation of the RTCWeb protocol stack
– SRTP
– ICE
– DTLS

• Allow sending and...
Media elements: HttpEndpoint
• Media downloading compatible with the HTML5 video
tag
– WebM (Chrome, Firefox)
– MP4 (Chrom...
Media elements: RtpEndpoint
• Full-duplex RTP multimedia exchange
–
–
–
–

H.264
H.263
VP8
Many different audio codecs sup...
Media elements: UriEndpoints
• PayerEndpoint
– Play media from file or URL
• Support for most popular formats

• RecorderE...
Media elements: Mixers (in progress)
•

Make possible group communications
– ForwardingMixer
•
•
•

One-to-many replicatio...
Filters
Seamless integration into OpenCV
Face recognition
Augmented reality
Subtitle adding
Color manipulation
QR detectio...
Filter with events
• Filters can provide events to the application
– Events are generated at the media server
– Events can...
The magic of pipelines: Transparent
media adaptation
• Agnostic media adaptor
– Acts every time a source is connected to a...
Complex examples: Heterogeneous
group communications
Media
Source

Media
Sink

Media
Sink

RecorderEndpoint

WebRtcEndpoin...
Complex examples: WebRTC to HTTP
Media
Source

Media
Sink

Media
Sink

RecorderEndpoint

Mixer

WebRtcEndpoint
Media
Sink
...
Complex examples: WebRTC Playing
Media
Source

Media
Sink

Media
Sink

RecorderEndpoint

Mixer

WebRtcEndpoint
Media
Sink
...
Kurento’s Roadmap
Q1 2014
• Installer and
documentation
• Group communications
• Super scalable media
repository

Q4 2014
...
Upcoming opportunities for
developers
• FI-WARE is the platform of the FI-PPP initiative
– European Commission FP7
– Kuren...
Collaborations welcome
http://www.github.com/kurento

Thank you very much for your attention
Complains, suggestions and co...
Upcoming SlideShare
Loading in …5
×

Kurento: a media server architecture and API for WebRTC

10,818 views

Published on

Introducing Kurento for WebRTC Expo 2013 (Paris). Kurento is an Open Source multimedia framework, which provides a Java EE compatible API suitable for adding real-time communication capabilities to any WWW application in a simple and seamless way.

Published in: Technology, Education
  • Be the first to comment

Kurento: a media server architecture and API for WebRTC

  1. 1. Kurento: a media server architecture and API Live video technology Luis Lopez lulop@kurento.org
  2. 2. Who am I? Associate Prof. • Computer Science • Multimedia protocols and systems 15 years of experience in multimedia applications and protocols research • Director of the FUN-LAB group Open Source Software enthusiast and contributor Coordinating the Kurento.org project : a media server architecture and API – http://www.kurento.org 2
  3. 3. What’s Kurento? Kurento • Esperanto translation of “stream” Esperanto philosophy • Openness • Simplicity • Interoperability Kurento • Open source development framework for creating rich real-time multimedia applications. : a media server architecture and API – http://www.kurento.org 3
  4. 4. Kurento’s history 2004 March 2013 • Different research efforts from private and public organizations in the area of multimedia • Kurento receives 4M€ by European Commission through different research initiatives 2012 December 2013 • The Kurento stack goes fully open source LGPL v2.1 • Kurento first release available : a media server architecture and API – http://www.kurento.org 4
  5. 5. Why WebRTC? • WebRTC is a movement – … by Tsahi Levent-Levi • Are RTCs new? – Skype, H.323, Facetime, WebEx • Why WebRTC is raising such expectations – Millions of WWW developers WWW dev. community RTC/IMS dev. community WebRTC WebRTC = Multimedia RTC is just another feature of your WWW application : a media server architecture and API – http://www.kurento.org 5
  6. 6. Why Kurento? WWW RTC developer experience Before WebRTC Developing the client side After WebRTC Begin Next natural step… End • • • • Begin • • • • Unified APIs Standards FOSS Multiplatform Developing the infrastructure side : a media server architecture and API – http://www.kurento.org End Unified APIs Standards FOSS Multiplatform Begin End • • • • Unified APIs Standards FOSS Multiplatform 6
  7. 7. Kurento media framework components •Low level media capabilities •Signaling and control capabilities •SIP, HTTP, RESFul, etc. •Java •Spring + JBoss + Mobicents SIP Servlet •Transport, encode/decode, transcode, CV, mix, etc •C/C++ •C++ Media Server + Gstreamer plugins Kurento Media Server Kurento Application Server Kurento Android SDK Kurento WWW SDK •RTC library for Android •Java and C •Multicodec support : a media server architecture and API – http://www.kurento.org •HTML5 multimedia development •Video tag, WebRTC •Javascript 7
  8. 8. Kurento Application Server: extending the WWW development model Signaling request: I want this media HTTP request: I want this resource Process WWW request - DDBB access - Transactions - Security tools - Etc. HTTP response: The resource Intuition behind traditional WWW Applications (Servlets, ASP, PHP, Rails, etc.) : a media server architecture and API – http://www.kurento.org Process media request - Media API - DDBB access Transactions Security tools Etc. Signaling response: The media is here Intuition behind Kurento development APIs: Multimedia RTC is just another feature of your application 8
  9. 9. Media API: media elements and media pipelines  Media Element • Provides a specific media functionality • Send/receive media Process media Transform media The Media API provides the capability of creating media pipelines by joining media elements of the toolbox The Media API provides a toolbox of media elements ready to be used. New media elements can be added Media Element PlayerEndPoint Sink SRC : a media server architecture and API – http://www.kurento.org WebRtcEndPoint Filter Sink SRC SRC • the desired media functionality. Building block Sink • › › › ›  Media pipeline • Chain of media elements implementing 9
  10. 10. Application architecture Other systems and databases Signaling and WWW traffic Media Media Signaling and WWW traffic HTTP Servlet SIP Servlet Web service s DD.BB. JMS Java EE compatible container Specific application logic Media API Decode Video Augmented Reality Video Playing and Recording Computer Vision Encode Video Kurento Media Server Infrastructure : a media server architecture and API – http://www.kurento.org 10
  11. 11. Possible use cases: just integrate with Java EE and GStreamer • Verticals – E-Health • Kurento + HAPI (http://hl7api.sourceforge.net/) – P2D video conferences as Electronic Health Records – Smart cities • Kurento + NGSI + OpenCV + Google Maps – City crowds movement tracking – Traffic density tracking • Telco infrastructures – Kurento + Mobicents • IMS application server • B2B & B2C WWW RTC – Kurento + CRM APIs • Enriched video conferencing with customer personal data – Kurento + ESB • Billing, video event processing, physical security, etc. : a media server architecture and API – http://www.kurento.org 11
  12. 12. Application example: requirements Kurento Media Server Infrastructure : a media server architecture and API – http://www.kurento.org 12
  13. 13. Application example: code @WebRtcMediaService(name = "MyWebRtcService", path = "/pathToService") public class MyWebRtcService implements WebRtcMediaHandler { public void onMediaRequest(WebRtcMediaRequest request) { //I can authenticate using any of the Java EE available mechanisms MediaPipelineFactory mpf = request.getMediaPipelineFactory(); MediaPipeline mp = mpf.createMediaPipeline(); //I could decide the type of processing connecting to a DDBB JackVaderFilter filter = mp.newFilter().withType(JackVaderFilter.class).build(); RecorderEndpoint recorder = mp.newRecorderEndpoint().withUri("file:///myFile.webm"); filter.connect(recorder); HttpEndpoint httpEndpoint = mp.newHttpEndpoint().build(); filter.connect(filter); //I could connect only audio or video separately request.startMedia(filter, filter); } To file Media Sink RecorderEndpoint Media Sink Sink Video tag or CDN SRC Media Sink HttpEndpoint : a media server architecture and API – JackVaderFilter http://www.kurento.org To Network Media Source From network WebRtcEndpoint 13
  14. 14. Application example: result : a media server architecture and API – http://www.kurento.org 14
  15. 15. Media element toolbox Transport Repository Group communications • WebRtcEndpoint • RtpEndpoint • HttpEndpoint • PlayerEndPoint • RecorderEndPoint • MainMixer • GridMixer • RoundRobinMixer Filters • FaceRecognitionFilter (events) • JackVaderFilter • QR/Barcode detector • PlateRecognitionFilter (events) • ColorTrackingFilter (events) And growing … Available as part of : a media server architecture and API – http://www.kurento.org 15
  16. 16. Media elements: WebRtcEndpoint • Full implementation of the RTCWeb protocol stack – SRTP – ICE – DTLS • Allow sending and receiving WebRTC flows at the media server infrastructure Media Source WebRTC traffic Media Sink WebRTC traffic WebRtcEndpoint : a media server architecture and API – http://www.kurento.org 16
  17. 17. Media elements: HttpEndpoint • Media downloading compatible with the HTML5 video tag – WebM (Chrome, Firefox) – MP4 (Chrome, Firefox, IE, Safari) • Media uploading compatible with HTML file input tag – Multipart support Media Sink HTTP WebM/MP4 HttpEndpoint : a media server architecture and API – http://www.kurento.org 17
  18. 18. Media elements: RtpEndpoint • Full-duplex RTP multimedia exchange – – – – H.264 H.263 VP8 Many different audio codecs supported Media Source RTP traffic Media Sink RTP traffic RtpEndpoint : a media server architecture and API – http://www.kurento.org 18
  19. 19. Media elements: UriEndpoints • PayerEndpoint – Play media from file or URL • Support for most popular formats • RecorderEndpoint – Record media to file or URL • WebM • MP4 Media Sink RecorderEndpoint Media Source PlayerEndpoint : a media server architecture and API – http://www.kurento.org 19
  20. 20. Media elements: Mixers (in progress) • Make possible group communications – ForwardingMixer • • • One-to-many replication of flows A source can be assigned to any of the sinks Multiple sources supported – MainMixer • • • Mixes media A source can be assigned to a combination of sinks Multiple sources supported Media Source Media Sink Media Source Media Sink Mixer Media Sink : a media server architecture and API – http://www.kurento.org Media Source 20
  21. 21. Filters Seamless integration into OpenCV Face recognition Augmented reality Subtitle adding Color manipulation QR detection People counter Plate recognition Etc. Filter SRC – – – – – – – – Sink • : a media server architecture and API – http://www.kurento.org 21
  22. 22. Filter with events • Filters can provide events to the application – Events are generated at the media server – Events can be propagated to the client app • Code example: MediaPipeline mp = mpf.create(); PlayerEndPoint playerEndPoint = mp.newPlayerEndPoint( "https://ci.kurento.com/video/barcodes.webm").build(); ZBarFilter filter = mp.newZBarFilter().build(); playerEndPoint.connect(filter); Events SRC Media Source Sink filter.addCodeFoundDataListener(new MediaEventListener<CodeFoundEvent>() { @Override public void onEvent(CodeFoundEvent event) { session.publishEvent(new ContentEvent(event.getType(), event.getValue())); ... PlayerEndpoint ZBarFilter : a media server architecture and API – http://www.kurento.org 22
  23. 23. The magic of pipelines: Transparent media adaptation • Agnostic media adaptor – Acts every time a source is connected to a sink – Adapts media formats as required by the involved media elements – 100% transparent for the application developer An agnostic media adaptor is “hidden” behind every connection making compatible element formats transparently VP8 H.264 : a media server architecture and API – http://www.kurento.org SRC Media Element Sink SRC Sink Media Element 23
  24. 24. Complex examples: Heterogeneous group communications Media Source Media Sink Media Sink RecorderEndpoint WebRtcEndpoint Media Source Mixer Media Sink WebRtcEndpoint Media Source SRC Media Sink Sink Filter RtpEndpoint : a media server architecture and API – http://www.kurento.org 24
  25. 25. Complex examples: WebRTC to HTTP Media Source Media Sink Media Sink RecorderEndpoint Mixer WebRtcEndpoint Media Sink HttpEndpoint Mixer Media Sink HttpEndpoint Media Sink HttpEndpoint : a media server architecture and API – http://www.kurento.org
  26. 26. Complex examples: WebRTC Playing Media Source Media Sink Media Sink RecorderEndpoint Mixer WebRtcEndpoint Media Sink Filter HttpEndpoint Sink SRC Media Source PlayerEndpoint : a media server architecture and API – http://www.kurento.org 26
  27. 27. Kurento’s Roadmap Q1 2014 • Installer and documentation • Group communications • Super scalable media repository Q4 2014 • Kurento cloud infrastructure • Hardware acceleration Q2 2014 2015 • WebRTC support on Android SDK • Kurento iOS SDK • Javascript Media API • Elastric scalable Kurento PaaS platform • SDN Kurento QoS cloud connectivity manager • IMS/Telco infrastructure integration • 3D/Kinect multimedia support : a media server architecture and API – http://www.kurento.org 27
  28. 28. Upcoming opportunities for developers • FI-WARE is the platform of the FI-PPP initiative – European Commission FP7 – Kurento is providing multimedia capabilities to FI-WARE • FI-WARE is giving 800K€ in prices to developers – You can opt to them developing on top of Kurento – http://www.fi-ware.eu/challenges/ • FI-PPP is providing 100M€ in funding to SMEs and entrepreneurs – http://www.fi-ppp.eu/how-to-participate/ : a media server architecture and API – http://www.kurento.org 28
  29. 29. Collaborations welcome http://www.github.com/kurento Thank you very much for your attention Complains, suggestions and comments can be sent to: Luis López lulop@kurento.org : a media server architecture and API – http://www.kurento.org 29

×