NUBOMEDIA is the first open source elastic cloud PaaS (Platform as a Service) specifically designed for real-time interactive multimedia services, which exposes its capabilities through simple APIs. It has been created on the top of Kurento technologies. This presentation provides an overview of NUBOMEDIA and details to create apps ready to be deployed on the NUBOMEDIA PaaS.
2. Table of contents
1. Introduction
2. NUBOMEDIA development
3. NUBOMEDIA PaaS
4. Let's get to work
5. Summary
3. Table of contents
1. Introduction
ā Review
ā What is NUBOMEDIA?
ā References
1. NUBOMEDIA development
2. NUBOMEDIA PaaS
3. Let's get to work
4. Summary
4. 1. Introduction
Review
āKurento is a media server and a set of APIs
aimed to create applications with advance media
capabilities
āThe building blocks for applications are named
media elements, chained into media pipelines
5. 1. Introduction
What is NUBOMEDIA?
āNUBOMEDIA is an open source PaaS
(Platform as a Service) based on Kurento
āNUBOMEDIA exposes to developers the
ability of deploying and leveraging
applications with media capabilities
7. Table of contents
1. Introduction
2. NUBOMEDIA development
ā Architecture
ā Media API
ā Kurento vs NUBOMEDIA apps
1. NUBOMEDIA PaaS
2. Let's get to work
3. Summary
8. 2. NUBOMEDIA development
Architecture
āThe Architecture in NUBOMEDIA is the same
than in Kurento
ā¢ Three-tier model (inspired in the Web)
Client
Application
Server
Media Server
NUBOMEDIA PaaS
9. 2. NUBOMEDIA development
Architecture
āLike every application with media capabilities, it is
important to distinguish between the media and
signaling plane
Client
Application
Server
Media Server
NUBOMEDIA PaaS
signaling
media
10. 2. NUBOMEDIA development
Media API
āKurento Media API allows to Java developers
consume the media services provided by Kurento
Media Server (KMS)
āConcepts:
ā¢ Media Element
ā¢ Media Pipeline
11. 2. NUBOMEDIA development
Media API
āKMS instances are provided elastically by NUBOMEDIA
ā¢ The number of available KMS instances depends on the PaaS
Manager configuration
āEach KMS has a total amount of available points to create
Media Pipelines and Media Elements
ā¢ The total points depends on the number of VCPUs of the KMS
ā¢ The type of the instance can be selected on the PaaS Manager
configuration
Instance type # VCPUs KMS points
Medium 2 200
Large 4 400
12. 2. NUBOMEDIA development
Kurento vs NUBOMEDIA apps
1.Maven dependencies <dependency>
<groupId>org.kurento</groupId>
<artifactId>kurento-client</artifactId>
</dependency>
<dependency>
<groupId>de.fhg.fokus.nubomedia</groupId>
<artifactId>nubomedia-media-client</artifactId>
</dependency>
<dependency>
<groupId>org.kurento</groupId>
<artifactId>kurento-client</artifactId>
</dependency>
In Kurento apps, we need the
kurento-client library to handle
KMS from the Java logic
In NUBOMEDIA apps, we need to
include also the nubomedia-
media-client to discover KMSs
within the NUBOMEDIA PaaS
13. 2. NUBOMEDIA development
Kurento vs NUBOMEDIA apps
2.Instance of KurentoClient
int sessionPoints = 25; // Chose a convenient value
for your pipeline
Properties properties = new Properties();
properties.add("loadPoints", sessionPoints);
KurentoClient kurentoClient =
KurentoClient.create(properties);
@Bean
public KurentoClient kurentoClient() {
return KurentoClient.create();
}
@Autowired
private KurentoClient kurento;
In Kurento, KMS is controlled by an
single instance of KurentoClient
(for example a Spring Bean)
In NUBOMEDIA, each new media
session should create an instance of
KurentoClient. The points
mechanism is used to locate or create
KMS instances (scaling in/out)
14. 2. NUBOMEDIA development
Kurento vs NUBOMEDIA apps
3.Deployment: Dockerfile
In Kurento, no Dokerfile is
required (deployment is managed
by hand)
In NUBOMEDIA, a Dokerfile
file is required
FROM nubomedia/apps-baseimage:src
MAINTAINER Nubomedia
ADD . /home/nubomedia
ENTRYPOINT cd /home/nubomedia && mvn spring-
boot:run
15. Table of contents
1. Introduction
2. NUBOMEDIA development
3. NUBOMEDIA PaaS
ā Introduction
ā PaaS GUI
1. Let's get to work
2. Summary
16. 3. NUBOMEDIA PaaS
Introduction
āThe NUBOMEDIA PaaS manager is a tool aimed to
control the way in which the NUBOMEDIA applications are
built and deployed inside the NUBOMEDIA PaaS
āThe capabilities provided by the Paas Manager can be
used by developers using the PaaS GUI:
ā¢ The PaaS Manager GUI is a web application that allows to use
the NUBOMEDIA PaaS Manager
17. 3. NUBOMEDIA PaaS
Introduction
āInternally, the NUBOMEDIA PaaS uses Docker
containers to deploy applications
āTherefore it is a requirement to include a Dockerfile in
GitHub repository to be deployed on NUBOMEDIA
āExample:
FROM nubomedia/apps-baseimage:src
MAINTAINER Nubomedia
ADD . /home/nubomedia
ENTRYPOINT cd /home/nubomedia && mvn spring-boot:run
https://docs.docker.com/engine/reference/builder/
18. PaaS GUI
āWeb application to manage NUBOMEDIA applications
3. NUBOMEDIA PaaS
http://paas-manager.nubomedia.eu:8081/
Credentials
needed to login
19. 3. NUBOMEDIA PaaS
PaaS GUI
āA NUBOMEDIA application can be deployed using the
PaaS GUI
āIt is done providing the GitHub repository URL and a set
of configuration parameters
21. Table of contents
1. Introduction
2. NUBOMEDIA development
3. NUBOMEDIA PaaS
4. Let's get to work
ā Introduction
ā Server-side
ā Client-side
ā Deployment
1. Summary
22. 4. Let's get to work
Introduction
āDevil is in the details, and so, we are going to see a
complete NUBOMEDIA step by step
āThis application is the nubomedia-magic-mirror tutorial
https://github.com/nubomedia/nubomedia-magic-mirror/
http://nubomedia.readthedocs.io/en/latest/tutorial/nubomedia-magic-mirror/
23. 4. Let's get to work
Server-side
āWe recommend Spring-Boot as the base technology for
NUBOMEDIA applications
ā¢ Spring-Boot embeds a Tomcat server in a simple seamless way
for developers
ā¢ The application is packaged as a runnable JAR, and when this
JAR is executed, the Tomcat server is started and the web
application automatically deployed
āWe recommend Maven as for managing the life-cycle and
managing the dependencies of our applications
24. 4. Let's get to work
Server-side
āThe first step is to clone GitHub repository
āWe recommend use an IDE to develop applications. For
example, Eclipse (importing app as Maven project)
git clone https://github.com/nubomedia/nubomedia-magic-mirror
25. 4. Let's get to work
Server-side
āFirst step is understand the pom.xml file:
<parent>
<groupId>org.kurento</groupId>
<artifactId>kurento-parent-pom</artifactId>
<version>6.6.0</version>
</parent>
<plugins>
<plugin>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-maven-
plugin</artifactId>
</plugin>
</plugins>
Inherit from kurento-parent-pom
makes simpler our pom.xml by
reusing versions defined in the parent
We can use the spring-boot
Maven plugin to simplify the
packaging as executable JAR
26. 4. Let's get to work
Server-side
<dependencies>
<!-- Spring -->
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-web</artifactId>
</dependency>
<dependency>
<groupId>org.springframework</groupId>
<artifactId>spring-websocket</artifactId>
</dependency>
<!-- Kurento -->
<dependency>
<groupId>org.kurento</groupId>
<artifactId>kurento-client</artifactId>
</dependency>
<dependency>
<groupId>org.kurento</groupId>
<artifactId>kurento-utils-js</artifactId>
</dependency>
<!-- Nubomedia -->
<dependency>
<groupId>de.fhg.fokus.nubomedia</groupId>
<artifactId>nubomedia-media-client</artifactId>
<version>${nubomedia-media-
client.version}</version>
</dependency>
ā¦
ā¦
Dependencies clause is one of
the most important. Notice that
our app depends on Spring,
Kurento, NUBOMEDIA and
WebJars (for the client-side)
ā¦
27. 4. Let's get to work
Server-side
āWe need to define also the way
in which signaling is going to
work in our application
āFor the shake of simplicity, we
use a raw WebSocket between
the server and client side
āWe need to define the messages
exchanged to carry out the
communication about client and
server
clientserver
start
startResponse
iceCandidate
onIceCandidate
stop
error
notEnoughResources
Starts a media
session
ICE candidates
(WebRTC
negotiation)
Stops a media
session
Error cases
28. 4. Let's get to work
Server-side
āThe class diagram of our application is the following:
MagicMirrorApp
MagicMirrorHandler
UserSession
Main class
Message handler
(signaling)
User session
(session id,
media logic)
29. 4. Let's get to work
Server-side
@SpringBootApplication
@EnableWebSocket
public class MagicMirrorApp implements WebSocketConfigurer {
@Bean
public MagicMirrorHandler handler() {
return new MagicMirrorHandler();
}
@Override
public void registerWebSocketHandlers(WebSocketHandlerRegistry registry) {
registry.addHandler(handler(), "/magicmirror");
}
public static void main(String[] args) throws Exception {
new SpringApplication(MagicMirrorApp.class).run(args);
}
}
MagicMirrorApp defines the
Spring-Boot application and
the WebSocket used for
signaling ("/magicmirror",
managed by
MagicMirrorHandler)
30. 4. Let's get to work
Server-side
public class MagicMirrorHandler extends TextWebSocketHandler {
private final ConcurrentHashMap<String, UserSession> users = new ConcurrentHashMap<>();
@Override
public void handleTextMessage(WebSocketSession session, TextMessage message) throws Exception {
JsonObject jsonMessage = new GsonBuilder().create().fromJson(message.getPayload(),
JsonObject.class);
switch (jsonMessage.get("id").getAsString()) {
case "start":
start(session, jsonMessage);
break;
case "stop":
release(session);
break;
case "onIceCandidate":
onIceCandidate(session, jsonMessage);
break;
default:
error(session, "Invalid message with id " + jsonMessage.get("id").getAsString());
break;
}
}
MagicMirrorHandler
manages signaling
messages. It keeps a
collection of user sessions
(thread-safe map users)
31. 4. Let's get to work
Server-side
public class UserSession {
public String startSession(final WebSocketSession session, String sdpOffer) {
// One KurentoClient instance per session (reserving points per session)
Properties properties = new Properties();
properties.add("loadPoints", POINTS_PER_SESSION);
kurentoClient = KurentoClient.create(properties);
// Media logic (pipeline and media elements connectivity)
mediaPipeline = kurentoClient.createMediaPipeline();
webRtcEndpoint = new WebRtcEndpoint.Builder(mediaPipeline).build();
FaceOverlayFilter faceOverlayFilter = new FaceOverlayFilter.Builder(mediaPipeline).build();
faceOverlayFilter.setOverlayedImage("http://files.kurento.org/img/mario-wings.png", -0.35F,
-1.2F, 1.6F, 1.6F);
webRtcEndpoint.connect(faceOverlayFilter);
faceOverlayFilter.connect(webRtcEndpoint);
// WebRTC negotiation
String sdpAnswer = webRtcEndpoint.processOffer(sdpOffer);
webRtcEndpoint.gatherCandidates();
return sdpAnswer;
}
UserSession
handles media logic
(media pipeline,
media elements,
WebRTC
negotiationā¦)
32. 4. Let's get to work
Server-side
āAs of Chrome 47, access to user media can only be done
by secure apps (HTTPS), and so, it is needed a Java
keystore
server.port: 8443
server.ssl.key-store: classpath:keystore.jks
server.ssl.key-store-password: kurento
server.ssl.keyStoreType: JKS
server.ssl.keyAlias: kurento-selfsigned
application.properties
http://doc-kurento.readthedocs.io/en/stable/mastering/securing-kurento-applications.html
33. 4. Let's get to work
Client-side <!-- WebJars -->
<dependency>
<groupId>org.webjars</groupId>
<artifactId>webjars-locator</artifactId>
</dependency>
<dependency>
<groupId>org.webjars.bower</groupId>
<artifactId>demo-console</artifactId>
</dependency>
<dependency>
<groupId>org.webjars.bower</groupId>
<artifactId>adapter.js</artifactId>
</dependency>
<dependency>
<groupId>org.webjars.bower</groupId>
<artifactId>jquery</artifactId>
</dependency>
<dependency>
<groupId>org.webjars.bower</groupId>
<artifactId>ekko-lightbox</artifactId>
</dependency>
</dependencies>
ā¦
Client-side dependencies are
handled by Maven
WebJars allows to declare
client-side dependencies (i.e.
CSS and JavaScript libraries)
also in the pom.xml. These
libraries are used linked in the
HTML client-side pages
34. 4. Let's get to work
Client-side
āThe app has been implemented as a SPA (single page
application) which contains a single HTML
<!DOCTYPE html>
<html>
<head>
<link rel="stylesheet" href="webjars/bootstrap/dist/css/bootstrap.min.css">
<link rel="stylesheet" href="webjars/ekko-lightbox/dist/ekko-lightbox.min.css">
<link rel="stylesheet" href="webjars/demo-console/index.css">
<link rel="stylesheet" href="css/styles.css">
<script src="webjars/jquery/dist/jquery.min.js"></script>
<script src="webjars/bootstrap/dist/js/bootstrap.min.js"></script>
<script src="webjars/ekko-lightbox/dist/ekko-lightbox.min.js"></script>
<script src="webjars/adapter.js/adapter.js"></script>
<script src="webjars/demo-console/index.js"></script>
<script src="js/kurento-utils.js"></script>
<script src="js/index.js"></script>
<title>NUBOMEDIA Tutorial: WebRTC Magic Mirror</title>
</head>
JavaScript/CSS are
linked here. Using
Bootstrap to provide
responsive design
35. 4. Let's get to work
Client-side
āA JavaScript file contains the client-side logic (message
handling and WebRtcPeer use)
var ws = new WebSocket('wss://' + location.host + '/magicmirror');
ws.onmessage = function(message) {
var parsedMessage = JSON.parse(message.data);
switch (parsedMessage.id) {
case 'startResponse':
startResponse(parsedMessage);
break;
case 'error':
onError("Error message from server: " + parsedMessage.message);
stop(false);
break;
case 'iceCandidate':
webRtcPeer.addIceCandidate(parsedMessage.candidate, function(error) {
if (error) return console.error("Error adding candidate: " + error);
});
break;
// ...
}
}
36. 4. Let's get to work
Client-side
āA JavaScript file contains the client-side logic (message
handling and WebRtcPeer use)
var webRtcPeer;
function start() {
var options = {
localVideo : videoInput,
remoteVideo : videoOutput,
onicecandidate : onIceCandidate
}
webRtcPeer = new kurentoUtils.WebRtcPeer.WebRtcPeerSendrecv(options, function(error) {
if (error) {
return console.error(error);
}
webRtcPeer.generateOffer(onOffer);
});
}
37. 4. Let's get to work
Deployment
āWe need a Dockerfile to deploy our app
FROM nubomedia/apps-baseimage:src
MAINTAINER Nubomedia
ADD . /home/nubomedia
ENTRYPOINT cd /home/nubomedia && mvn exec:java
38. 4. Let's get to work
Deployment
āFinally we can deploy our app in the NUBOMEDIA PaaS:
39. 4. Let's get to work
Deployment
āThe app changes its state, from CREATED to RUNNING (it
takes a couple of minutes to finish):
40. 4. Let's get to work
Deployment
āWe can trace two types of logs: build and app
I0708 12:51:02.335080 1 builder.go:41] $BUILD env
var is {"kind":"Build","apiVersion":"v1","metadata":{...}
I0708 12:51:02.423366 1 source.go:96] git ls-remote
https://github.com/nubomedia/nubomedia-magic-mirror
--heads
I0708 12:51:02.423547 1 repository.go:275]
Executing git ls-remote
https://github.com/nubomedia/nubomedia-magic-mirror
--heads
I0708 12:51:04.931566 1 source.go:189] Cloning
source from https://github.com/nubomedia/nubomedia-magic-
mirror
...
Image already exists
latest: digest:
sha256:24ac51ca448edfb1582f0bffc990c95506065f0aa26d5b295d
1cb32d35ce2dfe size: 49623
I0708 12:54:31.392578 1 docker.go:99] Push
successful
43. Table of contents
1. Introduction
2. NUBOMEDIA development
3. NUBOMEDIA PaaS
4. Let's get to work
5. Summary
ā Things to remember for NUBOMEDIA apps
44. 5. Summary
Things to remember for NUBOMEDIA apps
āFrom a high level point of view:
ā¢ The media-api concepts (media pipelines, media elements) are
exactly the same in NUBOMEDIA
ā¢ The NUBOMEDIA PaaS is going to provide elastically instances
of KMSs to our application
āFrom a low level point of view:
1. We need to add the nubomedia-media-client dependency
2. We need to create a KurentoClient instance per media
session in order to inform the PaaS when KMSs are required
3. We need a Dockerfile to configure the execution point of our
application