This report describes an approach of using a “Master-Slave” network communication mechanism by using Java Bindings for OpenGL API (Jogl), based on a real implementation, to achieve the effect of displaying 3D object within multiple screens.

1. Computer Graphics Lab
Multiple Screens
University of Trento
Mu Qiao (133734)
Supervisor: Dr. Giuseppe Conti
Prof. Raffaele de Amicis
-1-

2. Table of Contents
1. Introduction..................................................................................................................... 3
2. Related Work .................................................................................................................. 6
2.1 Studierstube............................................................................................................... 6
2.2 VRJuggler ................................................................................................................. 7
2.3 Java3D....................................................................................................................... 9
3. Methods and Implementation of Multiple Screens....................................................... 11
3.1 Development Tools, libraries and Platform ............................................................ 11
3.2 Rotating Cube Example .......................................................................................... 13
3.2.1 Class Diagram of Original Rotating Cube ....................................................... 14
3.2.2 “Master-Slave” Rotating Cube Extension ....................................................... 14
3.2.3 Network Communication through Java Socket ............................................... 17
3.2.4 Multiple Screens Effect Realization by Adjusting Viewport .......................... 21
3.2.5 Synchronization of “Slave” Applications ........................................................ 24
3.3 World Wind Java SDK ........................................................................................... 25
3.3.1 Extension of Original World Wind Java SDK................................................. 25
3.3.2 Network Communication through Java Sockets.............................................. 28
3.3.3 Multiple Screens Effect Realization by Adjusting Viewport .......................... 30
4. Performance Evaluation and Future Work ................................................................... 31
4. Summary ....................................................................................................................... 33
References......................................................................................................................... 34
-2-

3. 1. Introduction
In many circumstances, wider vision is needed to fulfill special requirements like reality
authentication of scenes (see Figure 1), commodity displayed in an exhibition, panorama
of a soccer match, exciting and shocking effect of computer games or simulation etc (see
Figure 2). A simple and costly way is to purchase a wider screen several times larger
than a normal one. An alternative is to utilize and combine multiple screens to achieve
the same effect. The advantage of multiple screens is the competitive cost, easy
transportation, free rearrangement over their relative positions. This report describes an
approach of using a “Master-Slave” network communication mechanism by using Java
Bindings for OpenGL API (Jogl), based on a real implementation, to achieve the effect of
displaying 3D object within multiple screens.
Figure 1: Reality authentication of beautiful scenes (from the website of 9x Media)
-3-

4. Figure 2: Computer Simulation of Aircraft (from bp2.blogger.com)
As part of the group project which aims to build a control station featuring in multiple
screens, advanced control system, sensors support and collaborative features, one of the
features described in this report is its multiple screens effect. By building a Java project
composed of both “Master” and “Slave” applications communicating through Java
sockets, the project is then planted over multiple PCs. One PC is running “Master”
application which controls several “Slave” applications running on other PCs. Multiple
screens effect is realized by synchronizing steps of “Slave” applications controlled by one
“Master” application.
The work begins with a relative simple application on a rotating 3D cube. The cube can
be controlled by both mouse and keyboard. The implementation results in a wider scene
composed of two screens through whose border the cube is able to “traverse”. Besides,
the size of the cube is magnified on two screens compared with the original one displayed
-4-

5. only in one screen. The steps of the rotating and moving of the cube is simultaneous over
the two “Slave” applications. After the rotating 3D cube application, further
implementation based on World Wind Java SDK from NASA is conducted in order to
achieve the similar result. With the much more complicated project structure of World
Wind compared with the 3D cube, many classes within the original project are extended
to achieve “Master-Slave” mechanism with good extensibility and transplantability.
The implementation of the multiple screens effect based on World Wind Java SDK can
be divided into steps below:
1. Identifying and extending classes which can be respectively inherited for different
use of “Master” and “Slave” applications as well as some of their ancestors and
offspring when needed. For example, classes of the drawing the earth, the
controlling of the earth and the registrations of these features are realized
differently from “Master” to “Slave” applications.
2. Realizing the “Master-Slave” communication through Java socket. “Master”
earth controls the orientation, rotation and size of the “Slave” earths by
establishing server-client communication. “Slaves”, treated as several different
servers, are listening to a “Master” which is treated as one single client. The
“Master” client controls several “Slave” servers by separately establishing
communication to each of the “Slaves” but sending the same command to all of
them in order to realize synchronization.
3. The sizes and positions of “Slave” earths need to be consistent to form a wider
view of a single earth within multiple screens. The screen size, position,
orientation, resolution as well as the viewport size of the scene display together
decides the relative sizes and positions among “Slave” earths.
This lab is focusing on the realization of multiple screens. This goal requires the
understanding of the graphic animation, network communication based on Jogl
implementation. The rest of the report is organized as below. Section 2 briefly
introduces other VR applications like Studierstube, VR Juggler and Java3D. Section 3 is
the core of the report beginning with the working environment of our application, and
-5-

6. mainly focusing on two multiple screens applications, one rotating cube example and one
based on World Wind application. Detailed explanation of the implementation procedure
is available through this part. In the end, this report ends with a summary as Section 4.
2. Related Work
2.1 Studierstube
The Studierstube augmented reality project, developed in Virtual Reality Group in
Technical University Vienna leaded by Prof. Dr. Dieter Schmalstieg, is one of the first
collaborative augmented reality systems in the world. Its innovative aspects are
represented by augmented reality, scientific visualization, multiple user support and
framework for easy extension and use [CBE].
Studierstube utilizes C++ to build on top of the Open Inventor (OIV) toolkit which
provides a higher layer of programming for OpenGL. OIV provides easy access to 3D
graphics and the core of it is an object-oriented scene graph scoring both geometric
information and active interaction objects [SFSEGP02]. OIV is extended when necessary
and form a toolkit of the OIV class hierarchy.
In the virtual environment, locales correspond to coordinate systems and coincide with
physical places [SFSEGP02]. Environment Management for Multi-User Information
Environments (EMMIE) [BHFMB99] developed by people from Columbia University
utilize one physical locale by all displays. This approach is simple to understand and
easy to implement. Studierstube uses a separate locale for each display in order to enable
all users having convenient access to all of the data items, especially with the increasing
data items [SFSEGP02]. Studierstube allows locales to be shared over displays. This
means though multiple displays can have independent points of view, they show images
of an identical scene graph (Figure 3). Usually, stationary displays can usually be
embedded in a single locale.
-6-

7. Figure 3: 3D Drag & Drop of An Application Object across Display Boundaries in
Studierstube Environment
2.2 VRJuggler
The VR Juggler project has been developed at the Iowa Center for Emerging
Manufacturing Technology (ICEMT) since 1997. VR Juggler is designed to “juggle” any
combination of devices, graphics APIs, and platforms that users want to use. VR Juggler
enables developers to create VR applications without knowing low-level details of VR
[JBBC].
VR Juggler is an application framework and set of C++ classes for writing virtual reality
applications. Developers are allowed to get direct access to various graphic APIs. The
hardware that VR Juggler supports includes CAVETM and C2, projection tables, simple
workstation monitor displays and traditional head-mounted displays [JBBC]. VR Juggler
currently runs on the Silicon Graphics, Inc. (SGI) platform and is being ported to Hewlett
Packard (HP) and Windows NT. For the multiple display effect supported hardware,
CAVETM, whose full name is Cave Automatic Virtual Environment, is an immersive
virtual reality environment where projectors are directed to multiple walls (see Figure 4).
The lifelike visual display is created by projectors positioned outside the CAVE and
controlled by physical movements from a user inside the CAVE. One of more computers,
often SGI workstation, drives the projectors. Techniques for rendering the scene include
OpenGL, OpenSG and OpenSceneGraph which are popular nowadays [Wiki-CAVE].
VR Juggler utilizes the power and flexibility of C++ and is thus an object-oriented
system. System details are handled by a set of Managers. For example, Draw Manager
deals with Graphics and Windowing API and communicates with Display Manager. All
-7-

8. the Managers are taken care of by a kernel on which user applications are based (see
Figure 5).
Figure 4: Virtual Reality - Multiple Walls of CAVE
Figure 5: Structure of VR Juggler
-8-

9. 2.3 Java3D
Java 3D is a scene graph-based 3D API for Java platform. It runs on top of either
OpenGL or Direct3D. Java 3D is not only a wrapper around graphics APIs, but an
interface that encapsulates the graphics programming using a real, object-oriented
concept [Wiki-Java3D].
1) Camera-Based View Model
Java 3D model is different from camera-based view model in the sense of binding the
scene model and view model together. The Java 3D view model incorporates head
tracking directly and if not, it acts very much like the camera-based view model.
Camera-based view model is controlled by three parameters: location, orientation and
perspective. The lookAt method within it creates a viewing matrix whose inverse can be
used to control the ViewPlatform object within the scene graph.
2) Structure of Java 3D View Model
Figure 6: View Object, Its Component Objects and Their Interconnection
Java 3D view model (see Figure 6) defines a Locale Object within a VirtualUniverse
Object which specifies the reference point within the universe. There are two branches
below Locale Object. The left branch groups define how scenes are rendered and
-9-

10. possibly navigated (ViewModel) and the right ones define the content of the scene
(ObjectModel). For the ViewModel, the TransformGroup Node under the right
BranchGroup Node specifies the position, orientation and scale of the ViewPlatform.
The transformed ViewPlatform defines the view of the user in the virtual universe. In
other words, ViewPlatform is the position of the viewer, it can be obtained and moved in
space by a TransformGroup. Besides, ViewPlatform is referenced by a View Object
which specifies all the parameters needed to render the scene from the point of view of
the ViewPlatform.
3) Physical Body and Physical Environment
Figure 7: Object and View Platform Transformations
Physical Body and Physical Environment (see Figure 7) under the View Object
encapsulates all the parameters associated with head, eyes and corresponding trackers. If
no trackers are available, they are ignored and ViewPlatform becomes the position of the
camera.
4) Multi-screens
- 10 -

11. Figure 8: A Portion of a Scene Graph Containing Three Screen3D Objects
Same View Object can be displayed in several screens from different perspectives. For
multi-screen environment (see Figure 8 and Figure 9), Java 3D must know how the
Screen3D are placed with respect to each other, the tracking device, and the physical
portion of the coexistence coordinate system.
Figure 9: A Three-Screen Display Environment
3. Methods and Implementation of Multiple Screens
3.1 Development Tools, libraries and Platform
Development tools:
• Java programming language
• Eclipse IDE(version 3.3.2)
- 11 -

12. Libraries:
• Jogl (version JSR-231)
• GlueGen
Platform:
• World Wind Java SDK (version 0.4.1)
Jogl is a wrapper library that allows OpenGL to be used in Java programming language.
Jogl [Wiki-Jogl]. Jogl is designed to provide harware-supported 3F graphics to
applications written in Java and full access to the APIs in the OpenGL 2.0 specification
as well as nearly all vendor extensions, and integrates with the AWT and Swing widget
sets [OpenGL]. GlueGen is a Java tool which automatically generates the Java and JNI
code necessary to call C libraries from Java code and it is originally developed for Jogl
[Wiki-Gluegen]. World Wind Java SDK, developed by NASA, enables developers to
embed World Wind technology in their own applications [NWW]. Besides, World Wind
Java SDK is developed based on Jogl which implies the necessity of inclusion of Jogl
library in our application (see Figure 10).
Figure 10: Application structure of our application based on World Wind Java which is
based on Jogl and JDK
- 12 -

13. 3.2 Rotating Cube Example
The implementation of a rotating cube is as a preparation for our goal application based
on World Wind Java SDK. The textured cube keeps rotating within a black background
and the view of the observer can be controlled by keyboard with “Up”, “Down”, “Left”,
“Right”, “PageUp”, “PageDown”, “A” and “Z”, respectively realizing the effect of
observer walking forward, backward, left and right, as well as looking up, down and
simply up, down. The desired aim of this implementation is to enable as observer
through controlling a “Master” cube by keyboard controls two “Slave” cubes in two
screens. Though there are two applications of “Slave” cubes existing, the final effect is
like only one larger cube being controlled within two screens (see Figure 11).
Figure 11: Two Screens Effect of A Rotating Cube
- 13 -

14. 3.2.1 Class Diagram of Original Rotating Cube
Class diagram of original rotating cube is drawn to assist analyzing and further
implementing “Master-Slave” mechanism upon (see Figure 12). The rotating cube is
drawn within GLRenderer, which implements GLEventListener, is added as GLCanvas’
listener. Besides, the instance of glRenderer is passed as a parameter to an instance of
KeyNavigator, extending KeyAdaptor, to implement the control of keyboards over the
rotating cube. The control is realized by modifying the parameter in an instance of
MovementStatus, an attribute of GLRenderer, while corresponding buttons are pressed.
The modification of these parameters will further change the view of a virtual camera
which represents the view point of the observer.
3.2.2 “Master-Slave” Rotating Cube Extension
In order to achieve the goal of controlling several “Slave” applications by one “Master”
application, the first step should be of extending original classes to “Master” and “Slave”
separately (see Figure 13).
The two classes that may need to be extended are GLRenderer and ApplicationStarter
because “Master” and “Slave” have different framework, drawing methods, what is most
important is their relationship as server and client (this will be specified in 3.2.3). For
GLRender, we change it to BasicGLRenderer which overrides the four functions “init”,
“display”, “displayChanged”, “reshape” because of its implementation of the interface
- 14 -

15. Figure 12: Class Diagram of Original Rotating Cube
- 15 -

16. Figure 13: Class Diagram of Multiple Screens Application of Rotating Cube
- 16 -

17. GLEventListener. Other functions of BasicGLRenderer are all abstract ones which are
implemented diffrerently by its offspring. The extensions of BasicGLRenderer are
MasterGLRenderer and SlaveGLRendere. They override the four functions “init”,
“display”, “displayChanged”, “reshape” again from BasicGLRenderer which implements
GLEventListener. The abstract functions in BasicGLRenderer are implemented
differently based on the requirement of “Master” and “Slave” applications. For
ApplicationStarter, extension of it is unnecessary because it only has one constructor.
Creation of class MasterApplicationStarter and SlaveApplicationStarter is proceeded as
needed. For example, MasterApplicationStarter includes MasterGLRenderer as an
attribute while SlaveApplicationStarter has SlaveGLRenderer. Other distinctions mainly
occur because of the network communication introduced in 3.2.3.
3.2.3 Network Communication through Java Socket
“Master” cube controls several “Slave” cube through network communication. For
example, button “Left” is pressed which moves the “Master” cube moves right (button
direction indicates the view position of the observer, “Left” means the observer moves
left, corresponding, object, the cube, moves right). The network communication is
realized in our application by sockets. Two classes of Socket and ServerSocket are used
for establishing the communication. Socket class implements client socket. A socket is
an endpoint for communication between two machines. ServerSocket class implements
server sockets. A server socket waits for requests to come in over the network. It
performs some operation based on that request, and then possibly returns a result to the
requester. We treat “Master” as client who has opened several sockets linked to different
servers, performed by “Slave” applications. Each “Slave” is seen as a server which opens
a serverSocket and keeps listening to specific port of the local address. Once client -
“Master” connects to the corresponding server, at the server side, a socket acting as a
client socket accepts what has been received from the client and then retrieves the objects
- 17 -

18. sent from the clients (see Figure 14).
Figure 14: Socket Communication between Server - “Slave” and Client - “Master”
Corresponding codes are illustrated and commented below:
a) Server - “Slave”
// open a new ServerSocket listening to a specified port
ServerSocket arrayServer = new ServerSocket(portNumReceived);
System.out.println("Server listening on port ." + portNumReceived);
System.out.println("Waiting for connections.");
// accept() blocks the caller until a correct connection is present
// an incoming connection is acknowledged and associated with an
// immediately created socket.
// The original socket is returned to the listening state
Socket client = arrayServer.accept();
- 18 -

19. System.out.println("Accepted a connection from: " +
client.getInetAddress().getHostName());
// new ObjectInputStream() recovers input stream from the client
ObjectInputStream ois = new ObjectInputStream(client.getInputStream());
// object1 reads serialized objects sent from the client
Object object1 = ois.readObject();
b) Client - “Master”
// specify the amount of servers the client is trying to establish
// communication with
int arraySize = 3;
Socket[] socket = new Socket[arraySize];
// try to build connection to the specified InetAddress and port
try
{
socket[0] = new Socket("localhost", 4000);
socket[1] = new Socket("192.168.253.20", 4001);
socket[2] = new Socket("192.168.253.21", 4001);
}
catch(Exception e)
{
System.out.println(e.getMessage);
}
Thus, by using sockets, network communication is established between server - “Slave”
and client - “Master”. Next step is to specify serialized objects that needed to be
transferred in order to realize instant control from “Master” to “Slave”. Objects that are
transferred through sockets need to implement Java Interface Serializable. Three
serialized objects are specified as ObjectEvent, ObjectStatusEvent, POVEvent which
includes information, respectively, regarding cube 3D positions, status (rotating or not)
and point of view from the observer. For the purpose of convenience and clearness, three
classes called SendObjectEvent, SendObjectStatusEvent and SendPOVEvent are built to
respectively send the three serialized objects mentioned above. Simple structures can be
seen as:
SendObjectEvent sends ObjectEvent
SendObjectStatusEvent sends ObjectStatusEvent
SendPOVEvent sends POVEvent
- 19 -

20. SendObjectEvent and SendPOVEvent happen during the display() function within
MasterGLRenderer class. This means that during each display, positions and point of
views are sent from “Master” to “Slave” to let the latter keep steps as precise as possible
with the former. SendPOVEvent also happens in the KeyNavigator class while keys are
pressed or released to transfer to current point of view of the observer.
SendObjectStatusEvent is the same case as SendPOVEvent here to command the status
of the “Slave” applications whether to start or stop rotating. The positions and point of
view of the rotating cube are set during the corresponding drawing period in the
MasterGLRenderer. It means the current values are updated in real time.
In the server - “Slave” side, after receiving corresponding objects (ObjectEvent,
ObjectStatusEvent, POVEvent) from the client - “Master” side, the first thing is to
distinguish which object it has received. Operator instanceof performs as the judger here.
After the identity of the received object is known, the corresponding set function can be
utilized in order to set the instant status of the SlaveGLRenderer. For example, if object
of POVEvent is received and after being recognized, the corresponding instance of
POVEvent within SlaveGLRenderer is set. Therefore, during moveCamera() function
within SlaveGLRenderer, instant point of view transferred from “Master” is immediately
set.
// within SlaveApplicationStarter, receive and specify the received
// object, pass it to the glRenderer
if (object1 instanceof POVEvent)
{
POVEvent povEvent = (POVEvent) object1;
glRenderer.setPOVPosition(povEvent);
}
// within SlaveGLRenderer, private member povEvent are set as what
// has been received
public void setPOVEvent(POVEvent povEvent)
{
this.povEvent= povEvent;
}
// command “Slave” to change its point of view according to the
// parameters bound within povEvent
private void moveCamera (GL gl)
{
gl.glRotated(povEvent.getLookupdown() , 1.0f, 0, 0);
gl.glRotated(povEvent.getSceneroty() , 0, 1.0f, 0);
- 20 -

21. gl.glTranslatef(povEvent.getXPos(),povEvent.getYPos(),povEvent.ge
tZPos());
}
Overall, objects containing parameters regarding the rotating cube are transferred through
sockets from the client - “Master” to the server - “Slave”. Parameters within serialized
objects are set immediately after they are generated. Then the objects are transferred
from “Master” to “Slave” and being recognized and passed to the renderer class within
which the parameters of the objects are released to their corresponding positions and
promote the movement of the “Slave”.
3.2.4 Multiple Screens Effect Realization by Adjusting Viewport
After establishing network communication between “Master” and “Slave” as well as the
transmission and set of the corresponding instant command issued from “Master” and
implemented by several “Slaves”, simultaneous and identical movement of “Slaves” are
realized from multiple screens (see Figure 15).
Figure 15: Simultaneous and Identical Movement of Two “Slave” Applications
- 21 -

22. In order to simulate the effect of only one cube within multiple screens, adjustment over
the “Slave” cubes are needed. Now, two problems regarding the adjustment are:
• Different sizes of cubes from different screens
• Different positions of cubes from different screens
a) The first problem “different sizes of cubes from different screens” is caused by the
differences of the resolution and sizes of screens. For resolution problem, if different
screens have the options of the same resolution, we can just easily set their resolution to
be the same. We implement our application in this way. If there is no uniform resolution
that every screen conforms to, ratios among different resolutions need to be calculated
and used as a factor to multiply the viewport parameters discussed below. For screen size
problem, a method to equalize cube sizes from different screens is to multiply the
viewport width and height with the corresponding ratios among screens.
In OpenGL, glViewport specifies the affine transformation of x and y from normalized
device coordinates to window coordinates. glViewport function is written as:
void glViewport ( int x, int y, int width, int height );
x, y specify the lower left corner of the viewport rectangle in pixels while width, height
specify the width and height of the viewport. Because the viewport size (width, height)
will determine the size of the cube displayed, if we multiply one “Slave” cube’s viewport
width and height with the ratio that expands it to the same size of a chosen standard
“Slave” screen, then the size of all “Slave” cubes become the same. For example, we
have “Slave” applications respectively displayed in screen A( width( A), height ( A)) ,
B ( width( B ), height ( B)) , C ( width(C ), height (C)) with the same resolution (we do not
consider the case of different resolutions). If we choose screen A as the standard screen,
width( A)
then ratios of B to A are widthRatio( BtoA) = and
width( B)
height ( A)
heightRatio( BtoA) = , the glViewport specified in “Slave” application B
height ( B)
becomes:
- 22 -

23. void glViewport (x, y, width( B ) × widthRatio( BtoA) , height ( B) × heightRatio( BtoA) );
(see Figure 16).
Similarly, the viewport C has to be modified the same way.
Viewport of B
Screen border of B
A B
width( A) width( B) width( B ) × widthRatio( BtoA)
Figure 16: Equalize cube size by adjusting glViewport
b) The second problem is to match the position of the already equalized cubes to simulate
the effect of only one cube is displayed and moved within multiple screens. The
adjustment of the position of cubes is also implemented within the glViewport. Due to
the consideration of conveniences, a class SliderPanel, extending JFrame and containing
two instance JSliders (to respectively adjust width and height value) is built to manage
the position of cubes, or more precisely, the viewport of the cubes. The slider.getValue()
is added to the position of the lower left corner of the viewport to move it by manually
adjusting the slider bar (see Figure 16).
c) Another problem that has not implicitly addressed is that the cube should be
independent of the change of the frame (the movement, resizing of the frame does not
influence the size and the position of the cube). This function’s realization would benefit
us from moving or resizing the frame at our will without damaging the multiple screens
effect. We achieve this by using the instantly origin of the frame with respect to the
screen and the origin of the viewport. For example, while the frame is moving down, the
distance decreased is added to the y value of the origin of the viewport, than the cube is
- 23 -

24. moving up with the same distance. This method will result the phenomenon that as if the
cube stays stable as before (see Figure 17).
Cube stays stable
Frame goes down
Viewport origin
Viewport origin moves up
Figure 17: Cube Size and Position are independent of Frame Adjustment
From the three steps described above:
I. “Master-Slave” Rotating Cube Extension
II. Network Communication Through Java Socket
III. Multiple Screens Effect Realization by Adjusting Viewport
The multiple screens effect based on network communication is achieved (see Figure 11).
By utilizing the techniques used in this cube example and adequate extensions, we further
implement the similar task based on World Wind Java SDK developed by NASA.
3.2.5 Synchronization of “Slave” Applications
Because of the differences of network transmission speed, different “Slaves” may act
unsynchronized after some time or under the condition of temporal network interruption.
If one “Slave” undergoes normal rotating while the other is experiencing network
interruption, then the phase of the former surpasses the latter. An idea to solve this
unsynchronization problem is to send serialized object (that is, sendObjectEvent()) for
every round of display() function. This is because the display() function keeps
- 24 -

25. circulating and displaying the rotating cube. If sendObjectEvent() is within the display()
function, then for every round of display(), the instant positions of the rotating cube is
sent through socket to different “Slaves”. While the display() goes to the next round, if at
least one of the former serialized object has not been consumed in the server - “Slave”
side, then the serialized object sent this time need to wait until the former serialized
object is accepted by all the “Slave” applications. In this way, if the delay of network
slows down one “Slave” application, all the other “Slave” applications are also dragged
behind but all of them are still synchronized. Adding sendPOVEvent() within display()
function also improve the synchronization effect.
3.3 World Wind Java SDK
World Wind is a free open source virtual globe developed by NASA and open source
community for use on personal computers [Wiki-NWW]. World Wind lets you zoom
from satellite altitude into any place on Earth [NAASA]. A preview of the World Wind
Java SDK was released in 2007 and the latest version 0.5.0 was released on May 2nd,
2008. The version that we are using is 0.4.1. With this SDK, we can embed World Wind
technology in our own applications – multiple screens.
3.3.1 Extension of Original World Wind Java SDK
Conceptually similar to what has been done for the rotating cube example, World Wind is
also extended respectively to client - “Master” and server - “Slave”. We, in most of the
cases, extend classes once necessary and put all the extended and newly generated classes
into one folder for the consideration of clearness and software reuse. We try not to
change the original code. But under special conditions, some slight changes of four
classes are made in our applications. They are:
• InputHandler: (functions added, for simplicity of coding)
o void setSocket(Socket[] socket);
o void setPOVEvent(POVEvent povEvent);
• AWTInputHandler: (functions added, because of its implementation of interface
InputHandler)
o public void setSocket(Socket[] socket){}
- 25 -

26. o public void setPOVEvent(POVEvent povEvent){}
• LatLon, Angle, Quaternion: (parameters need to be transferred through network,
for simplicity of coding)
o implements Serializable
• Configuration: (codes added, for the creation of new classes by class name)
o defaults.setProperty(MyAVKey.MASTER_WORLD_WINDOW_CLASS_NAME,
MasterWorldWindowGLAutoDrawable.class.getName());
o defaults.setProperty(MyAVKey.SLAVE_WORLD_WINDOW_CLASS_NAME,
SlaveWorldWindowGLAutoDrawable.class.getName());
o defaults.setProperty(MyAVKey.MASTER_INPUT_HANDLER_CLASS_NAM
E, MasterAWTInputHandler.class.getName());
o defaults.setProperty(MyAVKey.SLAVE_INPUT_HANDLER_CLASS_NAME,
SlaveAWTInputHandler.class.getName());
The above four illustrated are the original classes that has been modified within the whole
World Wind source code. Other than these four classes, the other original classes remain
the same. The extension of the original classes and their main differences are explained
below:
a) At the very top level, ApplicationStarter is replaced by MasterSimpleStarter and
SlaveSimpleStarter, both of which extends Thread. Then the SlaveSimpleStarter
can draw the earth at one thread, and listen and receive serialized objects at
another hand.
b) The original WorldWindowGLCanvas is replaced by
BasicWorldWindowGLCanvas which is further extended respectively by
MasterWorldWindowGLCanvas and SlaveWorldWindowGLCanvas.
BasicWorldWindowGLCanvas is seens as an abstract class. The main differences
between MasterWorldWindowGLCanvas and SlaveWorldWindowGLCanvas are
that the former uses the function setSocket (Socket[] socket) to further pass client
sockets to the associated class which has the controlling parameters, and the latter
uses the function setPOVEvent(POVEvent povEvent) to pass the received
povEvent from “Master” to the associated class which needs the controlling
parameters to act correspondingly as commanded. Both of them has the
- 26 -

27. setFrame(MyFrame frame) function to realized the effect of remaining stable
regardless of the moving and resizing of the frame (see Figure 17).
c) WorldWindowGLAutoDrawable, originally responsible for displaying the earth,
is extended to MasterWorldWindowGLAutoDrawable and
SlaveWorldWindowGLAutoDrawable. They perform differently mainly in
display(). The latter has the function setPOVEvent(POVEvent povEvent) to pass
the transferred object to the associated class. Besides, both
WorldWindowGLAutoDrawable and SlaveWorldWindowGLAutoDrawable
extends MyWorldWindowGLDrawable which extends
WorldWindowGLDrawable.
d) MasterAWTInputHandler implements InputHandler, KeyListener, MouseListener,
MouseMotionListener, MouseWheelListener, FocusListener to realize the mouse
and key event through its private member viewInputBroker.
SlaveAWTInputHandler does not need to implement these events since it only
carries out the command from the “Master”.
e) MasterOrbitViewInputBroker implements KeyListener, MouseListener,
MouseMotionListener, MouseWheelListener, FocusListener. This class is
associated with MasterAWTInputHandler and response to its corresponding call.
The functions in MasterOrbitViewInputBroker set the parameters related to the
control of either keyboard or mouse. From here, these parameters can be wrapped
to the serialized object povEvent and send through sockets to “Slaves”.
SlaveOrbitViewInputBroker is not necessary to implement the interface that
MasterOrbitViewInputBroker does.
f) Other extensions include MyAVKey extending AVKey which can be seen as an
index of the necessary class names of whose classes are called by name.
MyCGFrame simply extends JFrame.
g) Added classes include SendPOVEvent and POVEvent which are responsible for
sending controlling parameters through sockets. SliderPanel is used to manually
set the position of the viewport’s lower left corner in order to move the position of
the earth. DummySlave series are used to start SlaveSimpleStarter by passing the
name and port number into it.
- 27 -

28. h) Classes that are simply moved from the original source code are KeyPollTimer
and OrbitViewInputstateIterator without any change. The reason is that they are
both set as private classes (in the original source code) and are used during
implementation.
Detailed information regarding the hierarchy of “Master-Slave” realization can be
referred to the class diagram (see Figure 18).
3.3.2 Network Communication through Java Sockets
The concept of network communication through sockets is similar to that of the rotating
cube example (see 3.2.3). At the client - “Master” side, sockets are built at the
MasterSimpleStarter class and passes through MasterWorldWindowGLCanvas,
MasterAWTInputHandler till MasterOrbitViewInputBroker.
MasterOrbitViewInputBroker contains parameters that response to keyboard or mouse
event and are wrapped into povEvent. Through socket communication, povEvent is
transferred through network and received by the run() function implemented by
SlaveSimpleStarter which extends Thread class. SlaveSimpleStarter class then pass the
povEvent that it has received to SlaveWorldWindowGLCanvas, SlaveAWTInputHandler
till SlaveOrbitViewInputBroker. SlaveOrbitViewInputBroker unpacked the received
povEvent and set the parameters to their corresponding places, achieving the
synchronized drawing with the “Master”.
Sketched sequence diagram shows the process respectively from the “Master” side (see
Figure 19) and “Slave” side (see Figure 20). Connection between client - “Master” and
server - “Slave” is assumed already built.
- 28 -

29. Figure 18: Class Diagram of the realization of “Master-Slave” mechanism
- 29 -

30. Figure 19: Socket Passing in Client - “Master” Side
Figure 20: POVEvent Passing in Server - “Slave” Side
3.3.3 Multiple Screens Effect Realization by Adjusting Viewport
As is the same case with the rotating cube example, different sizes and positions of
“Slaves” are the main problems this step should solve. The realization method is quite
- 30 -

31. similar to that of the rotating cube example (see 3.2.4). Screen size ratios are calculated
and used to adjust the viewport size. SliderPanel class is created to adjust the position of
the lower left corner of viewports which equally means the changing of the earth inside
the viewport. glViewport exists in the class SlaveWorldWindowGLDrawable along with
the corresponding modification of its parameters. Besides, the problem of the size and
position of the earth is independent of the frame is also solved the same way that rotating
cube does. No matter how the frame is resized or moved, the earth inside it stays stable
(see Figure 17).
4. Performance Evaluation and Future Work
This lab generally realized the effect of multiple screens by socket communication.
Experiments on two “Slaves” are made and described here. First, two screens are placed
side by side as two “Slaves”. A third PC is used as the “Master” to issue commands by
utilizing keyboard or mouse. By issuing commands from the “Master”, two “Slaves”
conform to the order they have received and act accordingly. For the rotating cube
example, the two “Slaves”, when positions are adjusted adequately, act synchronized as if
they were one. This is mainly because the simplicity of in the code which enables precise
synchronization under frequent commands. The rotating cube is enlarged within two
screens and can traverse through the border of them (see Figure 11). For World Wind,
“Slave” applications are synchronized if actions performed on “Master” are not frequent.
If high frequent manipulation is operated on “Master”, then unsynchronization possibly
occur. The reason might result from the complexity of the code, especially in the
iterative calculation part. Because in order to reach the result of smoothly change under
manipulation, a class OrbitViewInputStateIterator is used to calculate the smoothly
change iteratively given an initial input parameter. The possible error during the iteration
might lead to differences among “Slave” applications since the parameters they have
received from the “Master” is only the initial input. A possible way to solve the
unsynchronization problem under high frequency manipulation on “Master” might be to
sendPOVEvent() during every round of the display() function which ensures the same
step of all the “Slaves”. This possible improvement on synchronization has not been
implemented yet.
- 31 -

32. As addressed in the body of the report, the positions of the “Slave” earths are adjusted by
setting the lower left corner of the viewport. This leads to a problem of “blank” area
within the screen, which means if the lower left corner is moved upper right, then the area
that is lower left than the new origin will block the scene since that area does not belong
to the viewport. The way to solve this problem might be moving the origin even lower
left as needed to erase the “blank” area existed in the screen. The rule needs to be obey is
to keep the viewport of all the “Slave” applications are the same assuming that their
screen resolutions are all the same.
An unfinished task is to include layers of the earth into the multiple screens effect.
Because of the separation of the earth itself and the layers (added as an listener),
synchronization of layers and its earth has not been achieved. The layers cannot be
moved by mouse manipulation as it is in the original source code.
Possible improvement includes the change of the orientation of “Slave” screens. Now,
screens are placed side by side and forming a straight line. In some circumstances,
certain angles between adjacent screens are necessary because it is quite possible to
generate better virtual effects (see Figure 3). Physical locations and orientations of
screens need to be calculated and added into the source code in order to realize such
effect.
- 32 -

33. Figure 21: Two “Slave” Earths and One “Master” Earth Displayed
4. Summary
The goal of this computer graphics lab is to realize multiple screens effects based on
World Wind Java SDK. Jogl is used as our development tool on which World Wind Java
SDK is based. Analysis on the source code, adequate extension of necessary classes,
network communication through sockets and viewport management are the mainstreams
during the lab process. In this report, we first present the related work done before in
order to obtain wider view in this field which includes Studierstube, VR Juggler and
Java3D. After that our work on multiple screens begins with the working environment,
including development tools (Java programming language, Eclipse IDE), libraries (Jogl,
GlueGen) and Platform (World Wind Java SDK), following by detailed descriptions
about one rotating cube example, seen as an introduction to the self-classified three steps
- “Master-Slave” Rotating Cube Extension, Network Communication through Java
- 33 -

34. Socket and Multiple Screens Effect Realization by Adjusting Viewport. Our final
application is based on World Wind Java SDK. Extensions on the source code is
wrapped for further software reusement. Corresponding Javadoc is provided as well.
References
[BHFMB99] Butz, A., Höllerer, T., Feiner, S., MacIntyre, B., & Beshers, C., (1999).
Enveloping users and computers in a collaborative 3D augmented reality. Proceedings
International Workshop on Augmented Reality (IWAR’99), 35– 44.
[CCBE] Jun Cai, Seminar Report: Augmented Reality: the Studierstube Project,
http://w5.cs.uni-sb.de/~butz/teaching/ie-ss03/papers/Studierstube/
[JBBC] Christopher Just, Allen Bierbaum, Albert Baker, Carolina Cruz-Neira, VR
Juggler: A Framework for Virtual Reality Development, Iowa Center for Emerging
Manufacturing Technology, Iowa State University.
[SFSEGP02] Schmalstieg, D., Fuhrmann, A., Szalavari, G. H. Z., Encarnacao, L.M.,
Gervautz, M., Purgathofer, W., The Studierstube Augmented Reality Project. In
Massachusetts Institute of Technology, Vol. 11, No.1, 33-54, 2002.
[NAASA] http://worldwind.arc.nasa.gov/
[NWW] NASA World Wind: http://worldwind.arc.nasa.gov/java/index.html
[OpenGL] http://www.opengl.org/resources/bindings/
[Wiki-CAVE] http://en.wikipedia.org/wiki/Cave_Automatic_Virtual_Environment
[Wiki-Gluegen] http://en.wikipedia.org/wiki/Gluegen
[Wiki-Java3D] http://en.wikipedia.org/wiki/Java_3D
[Wiki-Jogl] http://en.wikipedia.org/wiki/Java_OpenGL
[Wiki-NWW] http://en.wikipedia.org/wiki/NASA_World_Wind
- 34 -