Bootcamp WeBUI session 1 www.fiware.org

1. Synchronization GE
Bootcamp presentation

2. Rationale
● When creating a multi-user, real time Web application, two
things are commonly needed
○ The underlying data (or scene) model
○ The communication protocol for synchronizing the data
between participants
● Custom-building these for an application is often not cost-
effective; a generic solution can be preferable

3. Synchronization GE
● Contains a generic, lightweight scene data model
● Provides a real time two-way synchronization mechanism
between server and clients
○ Uses the WebSocket protocol for web clients
● Implementation based on realXtend open source virtual world
technology (www.realxtend.org) developed from 2007
onward
● Catalogue page: catalogue.fi-
ware.org/enablers/synchronization

4. Reference client and server
● Server and desktop client: the realXtend Tundra SDK
○ Core written in C++, JavaScript scripting
○ Relatively complex and heavyweight
○ github.com/realXtend/tundra
● Web client: Javascript
○ github.com/realXtend/WebTundra
○ Same codebase contains also the related 3D-UI
(WebTundra) GE

5. Web client demo
● N-player Pong game 130.206.81.111/pong/Pong.html
● Open multiple tabs/windows to join in more players
● Clients connect to a server, server sends scene changes (ball
+ bats moving) to all clients

6. Scene data model
● Based on Entities, Components and Attributes
● Using the Pong game example
○ The ball and bats are separate Entities
○ Entities contain components to separate functionality:
■ Mesh: visual representation
■ RigidBody: physics & collision
■ Placeable: positioning
○ Components contain attributes
■ For example Mesh has URL to the 3D model that
should be rendered, Placeable has position, rotation &
scale
● Application can define its own components

7. High-level view of the protocol
● When a new client connects, the server sends all entities to it
● After this, the following operations are automatically
synchronized both ways:
○ Create or remove an entity
○ Create or remove a component from an entity
○ Change an attribute value inside a component
■ This is commonly the bulk of network data
○ Send an RPC-like Entity Action
■ Has a string name and list of string parameters, can
be used for custom messaging, for example controls
handling in a virtual world

8. Protocol implementation
● Based on binary formatted messages for efficiency
● The protocol is described at
github.com/realXtend/tundra/wiki/Tundra-protocol

9. Running the server
● Typical command line to run Tundra without rendering, load a
specific scene file, and serve it to clients
./Tundra --headless --file <filename.txml> --server
● Optionally specify --port <portnumber> to select the port in
which to listen for clients (default 2345)
● On Unix machines without x11, use xvfb
xvfb-run ./Tundra --headless --file ...

10. Client code example
● Minimal JavaScript client code to connect to a server
var client = new Tundra.WebSocketClient();
var scene = new Tundra.Scene();
var syncManager = new Tundra.SyncManager(client, scene);
var loginData = {"username": "Test User"};
client.connect("localhost", 2345, loginData);
● Tundra.WebSocketClient manages the WebSocket connection to the server
● Tundra.Scene is the scene data model (initially empty)
● Tundra.SyncManager implements the synchronization protocol
● After successfully connecting, the SyncManager receives messages from the
WebSocketClient, and acts upon the Scene
○ As server sends the initial scene contents, and any changes, SyncManager mirrors
them to our local scene
● Note the default Tundra port 2345. This can be configured

11. Sending changes & scene signals
● SyncManager applies changes from server automatically as
WebSocket messages are received. To send client’s scene
changes to server requires an explicit call:
syncManager.sendChanges();
● Scene has change signals (using JS Signals library) which are
fired both for server- and client-side changes to the data:
scene.entityCreated
scene.entityRemoved
scene.componentAdded
scene.componentRemoved
scene.attributeChanged
scene.actionTriggered
...

12. Where to get + further resources
Links to guides & download links at: catalogue.fi-
ware.org/enablers/synchronization
Prebuilt server packages exist for Windows (32/64bit) &
Ubuntu Linux (32/64bit), can also build the server from
source (warning: may be involved)
github.com/realXtend/tundra

13. 3D-UI WebTundra GE
Bootcamp presentation

14. realXtend
Background: realXtend history
● Open source networked virtual world and game platform
(2007-)
● Originally started by consortium of companies and later on
joined by the University of Oulu, Finland
● Governed by non-profit realXtend Association
Stakeholders
● International user and developer base from research and
commercial sides
● Deployed applications used by several companies and
organisations in production

15. realXtend
Tundra
● Virtual world platform / 3D application framework
○ Server + Client sharing the same native code (C++)
○ Qt as main framework, Ogre3D for rendering
○ Multi-platform: Windows, Mac, and Linux
● Built-in efficient and versatile networking
● Uses Entity-Component-Attribute scene model
○ Automatic networking for components
● App development: Javascript or C++ for plugin extensions
● Import 3D assets from Blender, 3DS Max etc.

16. RealXtend Tundra based virtual world hosting service
● Distributed cloud hosting and asset storage
● One of the biggest contributors to the Tundra open source repositories
● Free plan for anyone to get their own 3D virtual world in minutes
● Growing fast -> over 3100 hosted worlds -> over 155 000 logins
Adds end-user functionality that the Tundra SDK does not address
● Web interface to manage your virtual worlds, user access, application settings
etc.
● Fast and easy Facebook / Google authentication or direct Meshmoon auth
● Web hosted asset storage access, importing 3D models, build tools, additional
entity-components, avatar editor, in-world applications, etc.
Check it out at www.meshmoon.com
Case: Meshmoon

17. Games

18. Presentations, art, showrooms etc.

19. Data visualization

20. Building industry

21. Education

22. ● Javascript library that uses three.js and Synchronization GE
to implement a Tundra client and WebGL rendering
● Asset loading, mapping three.js objects to Tundra EC:s,
handling networked changes to these in real time
● For a preview about what you can do with three.js, look in
http://threejs.org/examples/
WebTundra

23. ● Building web-based multiuser networked 3D apps
○ Or: Easy to use networking for three.js WebGL apps
● Client for existing Tundra scenes and applications,
interoperating with the native desktop Tundra client
● Both cases benefit from Tundra’s existing production-tested
networking system and scriptable server implementation
Use cases

24. Entity-Component system
● Replication of objects and changes to their state are handled
by Entity-Component system
● EC systems are a common way to use composition instead of
inheritance to compose things in 3D engines and games
● In this case the EC system also supports network sync, so
creating and modifying entities and their components can be
replicated over the network to the server
● More closely covered in Synchronization GE presentation

25. Anatomy of a basic WebTundra app
● Start from an existing three.js WebGL application
● Replication of objects and changes to their state are handled
by the network-aware Entity-Component system
● For each network-aware three.js 3D geometry object, an
entity with Placeable and Mesh components is created.
● Mesh component loads the 3D asset off the web and creates
a corresponding three.js mesh in the scene
● Changing Placeable attributes like position, scale, rotation are
synced to other clients via the server (and changed in the
corresponding three.js object).

26. Setting up
● Need to have a server to connect to, can use FIWARE Lab or
own machine, we also have a server running you can use
right away
● Set up your local development environment and download
WebTundra, or develop client on a FIWARE Lab vm - we have
an installation recipe

27. Pong example
● We’ve made available a multiplayer Pong game and covered
the internals in some detail in our User’s and Programmer’s
guide at http://catalogue.fi-
ware.org/enablers/documentation-21
● Browse source code at:
https://github.com/playsign/PongThreeJS/tree/ec
● Uses JQuery for 2D menus, server-side Bullet physics - more
“bells and whistles” than strictly necessary but aiming to be
a complete

28. Pong example
● Script running on Tundra server
creates U-shaped areas for each
player and receives bat movement
events from clients, and simulates
ball movements, sending resulting
entity movements to clients
● WebTundra clients present
synchronized entity state from
server and send events to server
based on player actions

29. Development tips and workflow
● Familiarize with three.js first
● Use Chrome/Firefox developer tools
● Watch Tundra server console
● Run server locally in graphical mode to see GUI for inspecting
entity state, triggering actions interactively, etc.
● Get help on realxtend-dev mailing list & #realxtend-dev on
irc.freenode.net

30. Real Virtual Interaction GE
Bootcamp presentation

31. Rationale
Need for open specification
● Every manufacturer have their own systems and those are not
compatible with other manufacturers systems
→ Objective was to create open source based infrastructure to be
used with next generation IoT applications

32. Real Virtual Interaction GE
● Real virtual interaction GE provides
o Services to interact with real objects (sensors) in the world
o UI components and associated (remote) services.
● Provides RESTful and WebSocket API’s for data requests
● Main GE components:
o Real virtual interaction server
o Reference RvI client
● (RvI is one of the key components for future IoT systems)

33. Possible use cases to utilize GE
● Monitor and control
o District heating system
o Bus locations
o Room temperatures

34. Key features
● Collect and store data from sensors
● Two directional data flow
● Real time data publication from server to client with WebSocket
● HTTP POST/GET request for third party services (for example to POI data provider)

35. Real Virtual Interaction Back-end Server

36. How to use GE
• Generic things for device application developers
• Data is sent over UDP
• Data should be formatted with RESTful data format specification
– format specification can be found from: http://forge.fi-
ware.org/plugins/mediawiki/wiki/fiware/index.php/Real_Virtual_Open_API_Specification
• Data may be compressed with gzip
• Device should publish its configurable parameters
• Configuration must be done with HTTP POST calls

37. How to use GE
● API example for web client application developers
http://serverUrl:port/
?action=
loadBySpatial
loadByID
device_id
&lon=
&lat=
&maxResults=
&radius=

38. Code samples for HTTP requests
● HTTP Get request: http://server_url:port/?action=loadBySpatial&lat=65.4&lon=25.4&radius=1500&maxResults=10
○ Sample form code
○ Sample JavaScript code
var xhr = new XMLHttpRequest();
xhr.open("GET", "http://server_url:port/?action=loadBySpatial&lat=65.4&lon=25.4&radius=1500&maxResults=10", true);
xhr.onload = function() {
console.log(this.response);
}
<form action="http://server_url:port/" method="get">
<input type="hidden" name="action" value="loadBySpatial">
Latitude: <input type="text" name="lat"><br>
Longitude: <input type="text" name="lon"><br>
Radius: <input type="text" name="radius"><br>
Maxresults: <input type="text" name="maxResults"><br>
<input type="submit" value="Submit">
</form>

39. How to control sensor
• Sample form to create POST call
• Sample javascript code to create POST call
<form action="http://server_url:port/upload" enctype="multipart/form-data" method="post">
Device id: <input type="text" name="device_id"><br>
Choose marker to upload: <input type="file" name="datafile" size="40"></br>
<input type="submit" value="Send">
function sendData(data) {
var xhr = new XMLHttpRequest();
var fd = new FormData();
for(name in data) {
fd.append(name, data[name]);
}
xhr.open('POST', 'http://server_url:port/upload');
xhr.send(fd);
}

40. Demo
acquire sensor data via RvI infrastructure
http://130.206.81.238:8080/rvi_client/

41. Where to get more information
FIWARE Wiki
• Real Virtual Interaction - Installation and Administration Guide
– http://forge.fi-ware.org/plugins/mediawiki/wiki/fiware/index.php/Real_Virtual_Interaction_-
_Installation_and_Administration_Guide
• Real Virtual Interaction - User and Programmers Guide
– http://forge.fi-ware.org/plugins/mediawiki/wiki/fiware/index.php/Real_Virtual_Interaction_-
_User_and_Programmers_Guide
FIWARE Catalogue
• Real Virtual Interaction
– http://catalogue.fi-ware.org/enablers/real-virtual-interaction

42. Thanks!Thanks!