This document discusses Kurento, an open-source WebRTC infrastructure and its APIs. Kurento provides a media server and client APIs that allow developers to create rich multimedia applications. The document outlines Kurento's modular client API, media processing capabilities like recording and computer vision, and higher-level APIs for group communications. It also describes how to develop custom media processing modules and provides resources for getting started with Kurento.
2. Who I am
http://www.kurento.org - lulop@kurento.org
2
Associate Professor
Escuela Técnica Superior de Ingenieros de Telecomunicación
Universidad Rey Juan Carlos (south Madrid)
Researcher in the area of RTC
• http://www.nubomedia.eu
• http://www.fiware.org
FOSS enthusiast
• Kurento project lead: http://www.kurento.org
• NUBOMEDIA Community lead: http://www.nubomedia.eu
3. Real-Time media Communications
http://www.kurento.org - lulop@kurento.org
3
Capture Encode Cipher Transport Transport Decipher Decode RenderNetwork
The media plane
The signaling plane
I wan to call you, do you accept?
Yes, do it in this way
4. WebRTC to the rescue.
Before WebRTC
First wave of
WebRTC technologies
Begin End
• APIs
• Standards
• FOSS
Development
experience
when working with
real-time media
Common WebRTC application (p2p communications)
WebRTC video stream
http://www.kurento.org - lulop@kurento.org
4
5. WebRTC infrastructures
http://www.kurento.org - lulop@kurento.org
5
Peer-to-Peer WebRTC Application (without media infrastructure)
WebRTC video stream
WebRTC Application based on media infrastructure
media infrastructure
6. WWW VS (Web)RTC applications
http://www.kurento.org - lulop@kurento.org
6
Control
Multimedia
Application logic
(developers’ code)
Multimedia Capabilities
Recording
Transcoding
Routing
Mixing
Analyzing
Etc.
Media
Traffic
RTC Media APIs
Multimedia Clients
Events
Application Signaling
Control
WWW
Application logic
(developers’ code)
Database Capabilities
Data storage
Data recovery
Data querying
Data processing
Etc.
DD.BB. API
WWW Clients
Events
Application Signaling
9. Kurento: a WebRTC infrastructure an
its APIs
http://www.kurento.org - lulop@kurento.org
9
Control
Multimedia
Application logic
(developers’ code)
Recording
Transcoding
Routing
Mixing
Analyzing
Adapting
Media
Traffic
RTC Media APIs
Multimedia Clients
Events
Application Signaling
Computer
vision
Augmented
reality
Blending
Etc.
Multimedia Capabilities
Kurento Media Server
(KMS)
Kurento Client API
Kurento Room API
Kurento Tree API
11. The Kurento FOSS Community
http://www.kurento.org - lulop@kurento.org
11
Community support WebRTC worldwide reference
International awards More than 300 companies
Very active mailing list
First result in Google
12. The Kurento FOSS project
http://www.kurento.org - lulop@kurento.org
12
• http://www.kurento.org
– Main web site
• https://www.twitter.com/kurentoms
– Main social channel
• https://groups.google.com/forum/#!forum/kurento
– Main mailing list
• https://github.com/kurento
– Main repository
• https://www.youtube.com/channel/UCFtGhWYqahVlzMgGNtEmKug
– Kurento Youtube channel
13. Developing with KMS:
The Kurento Client API
http://www.kurento.org - lulop@kurento.org
13
SinkSRC
Sink
SRC
SRCSink
Sink
Media Element
• Provides a specific media
functionality
› Send/receive media
› Process media
› Transform media
• Exchange media through
› Sources
› Sinks
Media pipeline
• Chain of media elements
implementing the desired media
logic.
• The Media API provides the
capability of creating media
pipelines by joining media
elements of the toolbox
Media Element
Sink
SRC
14. A modular API
• Modular in the sense of modularity*
– Isolation
• Internal states of a module don’t affect the rest
– Abstracion
• Internal states are hidden behind an interface
– Composability
• Interface must enable module recombination and assembly
– Reusability
• If a module has it, you can use it
– Extensibility
• If a module does not have it, you can add it.
http://www.kurento.org - lulop@kurento.org
14
* Baldwin, Carliss Young, and Kim B. Clark. Design rules: The power of modularity. Vol. 1. MIT press, 2000
15. The paradigm of modularity
http://www.kurento.org - lulop@kurento.org
15
16. The Kurento Client API in one word
http://www.kurento.org - lulop@kurento.org
16
sourceElement.connect(sinkElement)
connect
18. //MediaPipeline is a holder of media elements
MediaPipeline pipeline = kurento.createMediaPipeline();
//Create your media elements
WebRtcEndpoint webRtcEndpoint =
new WebRtcEndpoint.Builder(pipeline).build();
//Connect your media elements
webRtcEndpoint.connect(webRtcEndpoint);
http://www.kurento.org - lulop@kurento.org
18
Control
Multimedia
Application logic
(developers’ code)
Recording
Transcoding
Routing
Mixing
Analyzing
Adapting
Kurento Media APIs
Multimedia Clients
Events
Application Signaling
Computer
vision
Augmented
reality
Blending
Etc.
Multimedia Capabilities
//Where to render the local and remote streams
var options = {
localVideo : videoInput,
remoteVideo : videoOutput,
onicecandidate : onIceCandidate }
//Start media capture and communications
webRtcPeer =
new kurentoUtils.WebRtcPeer.WebRtcPeerSendrecv(
options,
function(error) {
if (error) return console.error(error);
webRtcPeer.generateOffer(onOffer);
});
Application Server Code (Java)
Client Server Code
Application source for Spring (Java EE)
Application source for Node.js (JavaScript)
19. Recording
http://www.kurento.org - lulop@kurento.org
SinkSRC
Sink
//MediaPipeline is a holder of media elements
MediaPipeline pipeline = kurento.createMediaPipeline();
//Create your media elements
WebRtcEndpoint webRtcEndpoint = new WebRtcEndpoint.Builder(pipeline).build();
RecorderEndpoint recorderEndpoint = new RecorderEndpoint.Builder(pipeline).build(
“file:///myfolder/myfile.mp4”);
//Connect your media elements
webRtcEndpoint.connect(webRtcEndpoint);
webRtcEndpoint.connect(recorderEndpoint);
19
Application source for Spring (Java EE)
Application source for Node.js (JavaScript)
20. Playing
http://www.kurento.org - lulop@kurento.org
20
SinkSRC
SRC
//MediaPipeline is a holder of media elements
MediaPipeline pipeline = kurento.createMediaPipeline();
//Create your media elements
WebRtcEndpoint webRtcEndpoint = new WebRtcEndpoint.Builder(pipeline).build();
PlayerEndpoint playerEndpoint = new PlayerEndpoint.Buildr(pipeline).build(
“file:///myfolder/myfile.mp4”); //RSTP and HTTP URIs also supported
//Connect your media elements
playerEndpoint.connect(webRtcEndpoint);
Application source for Spring (Java EE)
Application source for Node.js (JavaScript)
26. Creating your own modules
http://www.kurento.org - lulop@kurento.org
26
Kurento
IDL Compiler
JSON
Module IDL
Description
Java Java-
Script
cpphpp
Media API code Toolchain
Auto-generated code
Module
Implementation
cmake
Interface
Developer
(1)
(2)
(3)
27. Going up the API stack: the room API
http://www.kurento.org - lulop@kurento.org
27
createRoom
deleteRoom
joinRoom
leaveRoom
publisMedia
unpublishMedia
onParticipantJoined
onParticipantLeft
onMediaPublished
onMediaUnpublished
onMessage
See demo
28. Going up the API stack: the tree API
http://www.kurento.org - lulop@kurento.org
28
createTree
deleteTree
joinAsPresenter
joinAsViewer
leave
30. Where to start
http://www.kurento.org - lulop@kurento.org
30
• http://www.kurento.org/documentation
– Installation and administration guide
– Java tutorials for Spring and Java EE developers
– JavaScript tutorials for Node.js developers
– JSON-RPC protocol documentation
• https://github.com/kurento
– Main repository