Interactive Shared Educational Environment (ISEE): Design,
Architecture, and User Interface
Xiangming Mu and Gary Marchionini
School of Information and Library Science Interaction Design Laboratory, UNC-CH
The Interactive Shared Educational Environment (ISEE) is an advanced real-time
multimedia application that supports highly interactive collaboration and distance
learning activities within a heterogeneous network context. The ISEE not only takes full
advantage of fast LAN campus networks or Internet2 wide area networks by providing
peer-to-peer multicast support, but it also can be used in the less advanced settings of
home users. Media (e.g., digital video) are integrated into a desktop style interface in the
ISEE. The ISEE allows users to interact with live multicasts , a shared web browser,
shared video/audio with thumbnails for quick navigation, and text chat. Collaborative
work is supported via a shared time line across the multiple ISEE tools. For example,
each comment in the chat text panel is associated with a timestamp, which indicates at
which point during the video viewing (i.e., in what context) the comment was made.
Clicking that timestamp by another user will jump that user’s video player to the same
timestamp. One challenge for a collaborative distance learning (CDL) system is to
support a high degree of interaction between users and the video player due to the delays
associated with re-buffering the video. With pre-buffering and a novel collaboration
protocol, ISEE not only supports dynamic user-media interactions in real time, but also
guarantees synchronization across participants. The system interface and architecture
Collaborative distance learning (CDL) applications are the result of many years of
advances in computer network and computer technology. The number of Internet hosts
has grown from 4 in 1969 to more than hundreds of millions today. The connection speed
for the US backbone has grown from 50 kbps to several GBps . Personal computers
are powerful enough to easily process high quality multimedia information like real-time
videos. Multimedia information (e.g. images, graphics, and videos) is widely produced
and can be easily accessed through the fast growing Internet. Using multimedia,
especially video, to help distance learning is the main characteristic of current CDL
Collaborative distance learning is a process that allows participants (e.g. students)
to join in and form a learning group to exchange comments or multimedia information
(e.g. images). During the learning process, a mediator (e.g. instructor) can join and guide
the learning. Or it can be a process for several students to get together to overview the
previous classes (videos) and to prepare for an examination. If guided under a mediator
to watch a video, this type of learning is often referred as Distributed Tutored Video
Instruction (DTVI) or Tutored Video Instruction (TVI). Studies [2-5] have indicated that
the DTVI or TVI students outperform students who attended the live lectures in the
traditional classroom and students who watched the live streaming lecture individually, in
both the situations with or without tutors (mediators). The success of DTVI or TVI is
attributed to the collaboration when the students watch the video.
Collaboratively watching a video means a group of students share the same video
at the same time. Physically the students can stay together or in distributed locations. The
advantage of distributed collaborative learning is obvious: it increases the flexibility and
scalability. Students located hundreds or thousands miles away do not have to travel a
long distance to “form a group” and the number of attending students can be much more
than a traditional classroom’s occupation. The system we will discuss in this paper
belongs to this category.
The video being watched in CDL system can be synchronized or non-
synchronized. One characteristic for synchronized video watching is the so-called “what I
see is what you see”. In most synchronized CDL systems, every student in a study group
shares one copy of the video clip. Student can not control the video playback (pause,
restart, etc.) unless he/she gets the control token based on various rules of floor control
. With the token, the student can pause the video at a time and incur a discussion. All
other group members’ video players are also paused at the same time due to the
synchronizing token mechanism. On the contrary, Non-synchronized CDL systems allow
students watching and controlling their video independently without interrupting other
group members’ watching.
In addition to the video, other media or communication “channels” can also be
added to build a rich, multiple dimensions collaborative learning environment. Image the
Two UNC students, Mike in Mexico City and John in Asheville, are taking a class Multimedia
Communication via distance education. Dr. Hughes, the instructor, lectures online and incorporates a
variety of audiovisual materials supporting her lectures. John and Mike are required to watch a video
clip: Senses and Sensitivity during the class. From their respective homes they start their CDL system and
connect to the Internet. Immediately they see a live video from Dr. Hughes and they can also hear the live
lecture. Dr. Hughes completes the lecture part of class and opens the discussion phase. Mike asks a
question in the chat room of the CDL system:“ I am very interested in the structure of the human ear,
where can I find additional information?”. Dr. Hughes sees the message from Mike in her chat room
window. “I will give you a website”. After Dr. Hughes types this message in the chat room, she opens the
shared browser and input the URL. Immediately, Mike’s and John’s browser are also redirected to the
same URL as Dr. Hughes’. John also joins the discussion and sends a message to the chat room: “ Here is
a picture about the ear structure, http://www.bioanim.com/CellTissueHumanBody3/uhoMorf1lgws.html ”.
Mike sees the URL in the chat room and double clicks on it to load the picture into his browser. After a
while, Mike recalls an example about antennae in the previous part of the video that is related to the
current content . He drags his video slider control back to that point and plays the clip. Then Mike sends a
message to the chat panel: “look at this example, how can the antenna (ear) of the mouse send the
messages to their brain?” By looking at the time-stamp following by the sender’s name, Dr. Hughes
realizes that Mike is talking about something early in the video. Clicking on the timestamp on the chat
room text, the video player of Dr. Hughes immediately jumps back to the exact same timestamp as Mike’s.
After browsing the example, Dr. Hughes says : “it works this way..” . Finally, when the class is finished,
Mike, John and Dr. Hughes each save the annotations in the chat panel for later reference along with
their personal notes.
In this scenario, communication components like the video player, chat panel, and
shared browser are used combined to achieve an effective collaborative learning
environment. Each component contains one or more communication channels: the video
player provides visual and audio channels for both pre-recorded video and real-time live
streaming; the chat panel enables text communication; and the browser builds a hypertext
communication channel across participants.
One of the challenges for CDLs is to provide a service to enable students to
interactively browse video content while maintaining a state of synchronization in order
to collaborate with others. For example, a student can “pause” the video to take notes, to
ask a question or just go get a cup of coffee. When the student “resumes” the video after
a while, he/she can “synchronize” his/her video with other group members at will. The
synchronization part of the service is achieved in most collaborative CDL and Computer
Supported Cooperative Work (CSCW) systems by sharing one copy of the video
playback among all the participants (students). And the collaboration is achieved via the
floor control mechanism. On one hand, students will be reluctant to control (i.e. pause)
the video playback if they know all the other team members’ video player will also be
affected. On the other hand, some students may dominate floor control. The Interactive
Shared Educational Environment (ISEE) is an advanced real-time multimedia distance
learning system that allows students not only control the playback of the video without
affecting others, but also can synchronize with the group at any point.
Another challenge for collaborative distance learning is how to effectively
integrate multiple information channels into an efficient system. In the ISEE, a real-time
video streaming channel, a shared browser channel, a pre-cached video channel, and a
shared chat room channel are provided and integrated in a desktop style user interface
(Fig.1). Highly interactive collaboration and distance learning activities within a
heterogeneous network context are supported. Information is shared across these various
information channels in the format of message objects. With the novel concept of “smart
alignment”, these shared message objects allow students do “alignment” across channels.
For example, double clicks on the URL presented in the shared chat room can align the
shared browser and update it to the same URL.
This paper is organized as follows: first, related research for CDL systems is
reviewed. Next, the user interface as well as the various visual components in ISEE are
introduced. Then a list of design requirements and the system architecture are described.
Finally, innovative features of ISEE are presented and results from informal testing are
2. Related Work
In terms of application domains, the current state of distance learning (CDL) applications
can be grossly classified into four categories: general tools, educational tools, commercial
tools, and research tools.
2.1 General Tools:
A generic video player is implemented in such tools to support direct video
playback. Video clips can be downloaded via the network or distributed through CD, or
DVD. A number of such tools are available. Representative products include Microsoft
Windows Media Player (www.microsoft.com), RealOne Player from
RealNetworks(www.real.com), and Quicktime from Apple Computer Inc.
(www.apple.com). The main advantage of these tools is that they are easy to use and
provide control over basic functions such as pause, play, stop, and jumping backwards or
forwards. The disadvantage for such applications under the CDL context is there is no
support for communication and collaboration, for example, no video conferencing, chat,
shared browsing or shared whiteboard are implemented in these generic tools.
2.2 Educational Tools:
There are many production-level systems for using video in distance education.
Notably, Stanford-online (http://stanford-online.edu)  provides online access to pre-
recorded video of live courses. This technology is often referred to as on-demand
streaming. Using this tool, the content of the video can be played by distance students via
Internet connection. A synchronized slide displayer is also provided. Slides or
screenshots of the scripts on the blackboard are synchronously displayed with the class
video. Choice of video qualities is available for different network connection speeds.
Other services such as digiscript (http://library.digiscript.com/) provide similar services
for conferences and professional meetings. The advantages for such tools are ease of use
( no extra installation except a network connection), choice of connection speeds, and
multiple information channels (visual, audio, and images). The disadvantage is problems
for scalability (the total bandwidth on the server is limited), long delays for each
reposition of the video playback (it takes time to re-buffer), and no collaboration
services (real-time chat-room, shared browser, or shared whiteboard) are supported.
2.3 Commercial Tools:
There are many commercial tools in the market that can be used for the purpose
of CDL [7-8]. In terms of the media transmission technology, these tools can be roughly
classified into two sub-categories: one that only supports one-way steaming, the other
that supports two-way or multiple way communication.
Representative products for one-way streaming includes RealONE Player
(www.real.com), QuickTime (www.apple.com), Microsoft Media Player, Netshow and
Vxtreme (www.microsoft.com). In the context of CDL, the one-way steaming can be
utilized to broadcast either real-time lectures, or on-demand video streaming. The
advantages for this kind of tools are their scalability and flexibility. A great number of
users can receive the streaming lecture simultaneously with no restriction on the
connection number. The disadvantages are no supports for two-way communication, and
no collaboration among group members.
On the market, there are various video conferencing and Internet meeting
software, including: Intel video Phone(www.intel.com), White Pine’s CU-
SEEME(www.cuseeme.com),Meeting Point, and Microsoft’s NetMeeting
(www.microsoft.com) which supports real-time two-way or multiple way
communication. As they are not specifically designed for CDL, the majority of the tools
in this sub-category do not provide specific supports for collaboration and information
sharing across students. For example, functions like integration of multiple information
channels are not supported in most systems. Furthermore, scalability and adaptability are
two main challenges. Limitations include: how to choose a videoconference system to
adapt to various network connection speeds to support a large number of students to join
in from distributed locations; how to allow students take full control of their video player,
as well as synchronizing with other students and allowing students to save the logs or
annotations for later use.
2.4 Research Tools:
A number of tools or prototypes are available as part of larger research and
development efforts. Some of the tools are MBONE tools such as vi (www-
nrg.ee.lb1.gov/vic) and wb (www-nrg.ee.lb1.gov/wb), IRI  ,IRI-h, and virtual
classroom systems . Multicast is widely used in the design of the research tools.
Multicast dramatically reduces the bandwidth consumption in the Internet, especially for
high quality real time applications like the multimedia distance learning systems.
Compared with multiple Unicast transmissions, the network latencies can also be highly
reduced by adopting Multicast. The disadvantage for research tools is that more works
are needed to make the tools and prototype more robust , more stable, and more usable.
2.5 Interactive Shared Educational Environment (ISEE)
Although we can find various advantages in the applications discussed above and
each of them offer a solution toward distance learning with video, the following functions
are rarely covered, let alone integrated:
1. Provide students a system that supports multiple views of the video in the CDL
context; allow students to preview the video or related annotation before a CDL
class; allow students to view the video and make annotations in real-time; and
allow students to overview videos and make annotations on videos from previous
2. Coordinate and integrate independent communication channels (video, audio, text
chat, web browser) into a uniform user interface. Information can be shared
information across these channels to achieve optimal system performance. For
example, a timestamp appear in the chat room is linked to the video player’s
timestamp and a single click on the timestamp in the chat room can update the
video player to the same point.
3. Provide students the services for both interactively controlling the video player
and synchronizing with other team members.
4. Accept and support metadata (like MPEG-7 compatible data in XML format) for
the video and effectively present the metadata in the user interface.
The Interactive Shared Educational Environment (ISEE) is a prototype
application that implements all the above functions.
3. User Interface
ISEE provides a desktop style Graphic User Interface (GUI) that integrates multiple
communication components (Fig.1). In the ISEE, students (or instructors) can rearrange
the layout of the GUI at will according to different learning contexts and personal
preferences. The positions of various components can be rearranged by dragging and
dropping and their size can also be changed (Fig.3). This is particularly useful when
multiple communication components are open simultaneously in limited screen real
estate. Such technology also provides a way to take full advantage of the high resolution
of large screens. For example, students can enlarge the relative size of a video player in a
high-resolution (1600*1248) monitor to get a better video display rather than use the
default size, which is designed for a screen resolution of 1024*768.
Another reason we choose a desktop style user interface for ISEE is the consideration of
information flow management. With various information channels (text, images, multiple
video and audios and hypertext), the issue of information overloading arises. A desktop
with internal frames to contain alternative information channels will provides users
(students) the choice to filter these multiple information flows by opening or closing the
Pre-cached Interactive User name and icons
Video Player (IVP): for Current Users
Video Player (SVP)
Shared Web Tooltip: indicate the
current timestap and Shared Chat Room (SCR) :
Storyboard for video with annotation and
clip in the IVP timestamp followed by each
Figure 1: Overall interface for ISEE
3.1 Streaming Video Player (SVP)
For any real-time multimedia distance learning system, the key component is the video
player that can receive streaming video sent from the instructor or a live stream sent
directly from the classroom. In ISEE, after students successfully log in with the correct
user name and password (Fig. 2), a button entitled “Live Streaming” is enabled and
students can thus press the button to begin receiving streaming video multicasted from
the remote server. An internal frame with a video player will pop up at the top-left of the
interface with the same size of the sending streaming video (Fig.1). Students can
determine the size by rearranging the frame’s size. A control panel is also presented
beneath the Streaming Video Player (SVP) that allows students to pause/resume the
stream, to adjust/mute the voice volume, and browse the media properties (media source,
video/audio format, video encoding type, frame rate, and current time-stamp).
Figure 2: Log in to the ISEE
3.2 Interactive Video Player (IVP)
One of our primary goals is to support usage of interactive video by students as
they study. A second window/channel is provided in ISEE for local control of a pre-
cached video on the client machine. Students not only can pause/resume the video
playback, but also can determine what happens next. For example, students can jump
forward/backward to an arbitrary point of the video and begin playing from that point, or
they can fast-forward the video to quickly browse the content of the video. In some cases
students need to pause the video and discuss with other students or ask questions to an
online instructor when they come across difficulties to understand the video (an abstruse
formula or a perplexing concept). For a synchronized steaming video player, pausing the
player will lose the remaining stream up to the point it is resumed. This is not what
The Interactive Video Player (IVP) is provided in ISEE to allow students full control of
the video player without losing information. Pressing on the resume button will resume
the video at the same point the video was paused. The IVP can also support the
synchronization video playback among members in a group.
Figure 3. Example for re-arranging the size of IVP in the ISEE interface
After selecting a video clip from the drop-down list of available pre-cached video clips in
the top panel of the ISEE interface (Fig. 4), the IVP will be presented in the center of the
ISEE and begin to play back. Below the IVP is a storyboard that contains a series of
frames, which refer to the start points of segments in the video clips (Fig.1). Single
clicking on one of the frame in the storyboard will update the IVP to the same timestamp
of that selected frame and begin the playback from there (like the functions provided by
DVDs to let users select a segment to begin to play). At the same time the Shared Web
Browser (SWB) will be updated to display a web page related to the video clips, such as
an outline of the video (Fig.4). The more detailed introduction about SWB is provided
next. A similar video control panel is also provided for IVP that can not only allow
students to pause/resume or stop the video, but also allow student to jump
forward/backward to an arbitrary position of the video by dragging the scroll bar beneath
the video player (Fig.1). Students can even drag the scroll bar to quickly navigate the
content of the video. We are investigating other surrogates in addition to the storyboard
in the IVP. A tooltip along with the scroll bar indicates the corresponding time-stamp.
An important innovation in the ISEE is the provision of the “synchronization”
function for the IVP. When a student logs in to the ISEE, he/she will join a virtual
learning group. Usually a mediator will also join the group to guide the study, but this is
optional. The person who established the session serves as the mediator. The mediator
can be a class instructor or a teaching assistant depending on the learning context. By a
single click on the “synchronize” button, which is located on the top of the ISEE interface
with the icon of an envelop and an arrow on the envelop (Fig. 1), users are able to
synchronize their IVP with the IVP of the mediator. For example, in a virtual class
scenario, the instructor acts as the mediator. Students join the class virtually from
distributed locations and open their own IVP to watch the video playback. For students
who join late, pressing the “synchronize button”, will immediately synchronize their IVP
with the instructor’s to share the same video timestamp. That is the so-called “what I see
is what you see”. During the class, students can navigate the video content by dragging
the scroll bar back and forth, or jumping around by clicking on the frames listed on the
storyboard, or pausing the IVP to take notes. Doing any of these operations results in the
non-synchronization of their IVP with others. Clicking on the synchronize button “brings
them back” immediately and synchronizes with the IVP of mediator.
Outline for video clip Select a video clips
(HTML format) available
Figure 4: Example for displaying a class outline in the SWB after the video clip for that class is
selected (Senses and Sensitivity lecture 1)
3.3 Interactive Chat Room (ICR)
Chat rooms are now widely used online as a media for virtual communication. In
ISEE, an Interactive Chat Room (ICR) is implemented (right bottom in Fig. 1). The
student can post messages in the ICR to join a discussion on a specific topic, or can
passively receive messages posted by other class members.
One important characteristic for ICR is linking the chat discussion to other
information resources. As with hyperlinks in an HTML page, a link mechanism is
provided in the ICR that coordinates display in the IVP and shared web browser. Clicking
one of the links in the ICR will update either the HTML page of the shared browser, or
the current timestamp of the IVP. For example, consider a student, John, taking a class on
web design. While watching the video “tutorial about web design” at 12min 33secs into
the video, John posts a question “ how do I change the color of the font in this example?”.
The message will appear on the ICR window of all the class members as follows:
how do I change the color of the font in this example?
Suppose at this point, the instructor’s IVP timestamp is 23min, 22secs and the topic in the
instructor’s IVP is “how to insert an image into your web page”. Most students are
synchronized with the instructors’ IVP and have no idea to which “example” John refers.
However, by double clicking the hyperlink timestamp (12:33) associated with John’s
posting, another student, Sue, can immediately jump his own IVP to the timestamp of
12min,33secs and begin to play from that point . Sue now knows what John is talking
about and she can post her response in the ICR “ you can…..”. After she is finished, Sue
can come back to “re-join” the class by clicking on the “synchronize” button.
ICR also supports hyperlinks. Any messages posted in the ICR that contains a
substring started with “http://; HTTP://; WWW. ; or www.” is added asa hyperlink
automatically to that substring . For example, if Sue sends a message that contains a
URL, it will be presented like this:
You can get more information about ear structure at http://www.sbu.ac.uk/~dirt/
By double clicking the underscored URL displayed in the ICR, the referred web page will
be presented in another component of the ISEE—the Shared Web Browser.
3.4 Shared Web Browser (SWB)
It is important that students can not only share text messages in the chat room, but
also share other information media like images, pictures, or even web pages. A
component called the Shared Web Browser (SWB) is supported in the ISEE (lower left in
Fig.1). SWB is a web browser style component that can present hypertext web pages
written in HTML. Hyperlinks are also enabled in SWB. Beneath the SWB in the same
internal frame, a text field is provided that allows students or instructors to input URLs
which will be propagated to all other students who joined the same study group or virtual
class. That means all the group members are able to share hypertext web pages via SWB.
For instance, the instructor can present class slides via SWB and students can exchange
music, images, etc.
3.5 Other widgets
3.5.1 Login widget:
This is a widget that provides user authorization to ISEE sessions. Before the user
is authorized, buttons for chat and live streaming are disabled and no information about
other users is presented.
3.5.2 User profile widget:
Awareness of other group members is important for a virtual class. So a user
profile frame is provided which lists all the users’ login names and their preferred image
icons (upper right in Fig. 1). The view for this widget is dynamically updated and always
displays the current members in the study group. We are investigating adding more
metadata information about the user in the next iteration of the ISEE. For example, the
number of messages posted to the ICR by a particular user.
3.5.3 User control panel
At the top of the ISEE interface, a set of buttons are provided which allow users
to open or to re-open the various components mentioned above. The “synchronize”
button also resides in this panel. Additionally, each internal window can be minimized,
maximized, or closed just as in other GUI applications.
4. Design requirements and system architecture
With a number of CDL tools available, the initial motivation for us to design the
ISEE was the absence of systems that coordinated information resources into a seamless
4.1 Design requirements for the ISEE
The ISEE system can be used in various tasks in CDL. It supports real-time live
class Multicast, ad hoc study sessions (with multiple class videos available) controlled by
students preparing for an examination or collaborating on a project, and virtual classes
using pre-recoded lectures (videos) under the mediation of a Graduate Assistant.
Another kind of flexibility provided by ISEE is the support of heterogeneous
network settings. The ISEE communication protocol we defined (discussed later), uses
only text messages to transmit among group members if the video is pre-cached and the
Multicast channel is disabled. For slow network connections (i.e. phone line), the video
can be obtained through other approaches, such as the distribution of CD instead of
directly downloading from Internet or downloading in the background while other tasks
are foregrounded. A peer-to-peer Multicast is also supported in ISEE to take full
advantage of the advanced Internet connections like Ethernet or Internet2. In these cases,
video clips can be dynamically updated by directly downloading from the server (Class
ISEE is not another video player passively receiving streaming video and playing
it the beginning to end with few controls. ISEE supports two distinct classes of
interaction: a person can use the information resources in private mode, thus interacting
with video or text individually, and a person can interact with other people mediated by
shared resources. The advantage of the ISEE is that users can move easily across these
types of interaction, using some information resources locally and sharing others. Figure
5 summarizes these interaction features. We believe that the integration of these private
and shared uses will lead to emergent new capabilities for students and teachers.
Resource/Function Private Shared
Start/stop X X
Audio/mute X X
Start/stop X X
Audio/mute X X
Jump X X
Annotate X X
View notes X* X
Take notes X* X
Search X X**
Browse X X**
Figure 5. ISEE Private and Shared Interaction Functions
Note: * private notes can be made in a client-side word processor. ** shared URLs are
passed through the chat resource.
ISEE can be used by a large number of distributed users. Using Multicast
technology for the real-time streaming, ISEE greatly enhances the scalability by just
sending one copy of the streamed content. The architecture (discussed later) is a peer-to-
peer structure in general instead of client/server structure. So there are fewer demands on
the server’s processing and bandwidth resources.
Java and Java compatible APIs were used to develop the system. ISEE is a pure
Java application and can be directly deployed on various operating system like Windows,
Unix, Linux, or Solaris, and on various platforms like IBM compatible PCs, Macintoshes,
or Solaris workstations with only minimal modifications. This will save time and effort
for extending, re-developing, and adapting the source code.
Based on the Real-time Transport Protocol (RTP), Multicast streaming is used for
real-time video. Currently only one pair of Multicast video/audio channels from the
mediator to students are supported in ISEE. It will be straightforward to extend this
design strategy to a peer-to-peer architecture in advanced networks such as Internet2.
Metadata, also called ‘data about data,’ provide information and descriptions on
the content of the video. In ISEE, we include metadata on video properties, video
summary ( keyframes displayed in the storyboard that presenting segments of the video),
tooltips for the key frame, and video outline in hypertext HTML format. Even though
there are multiple choices like Dublin Core(http://purl.org/DC), The Society of Motion
Picture and Television Engineers (SMPTE), Metadata Dictionary (www.smpte.org),
OpenSMEF from Standard Media Exchange Framework(SMEF), INDECS
(http://indecs.org), we decide to use the MPEG-7 as our metadata standard. XML is
adopted to define and to describe the definitions and definition schemas based on
MPEG-7. Some of the reasons we choose MPEG-7 are: it is media independent, it is
supported from both the academic and commercial sectors, and it has a rich set of
descriptions. And more importantly, such a metadata schema provides the data
interoperability for our ISEE system.
4.2 System architecture:
4.2.1 Java Shared Data Toolkits (JSDT)
In ISEE, a Session Sever (SS) is implemented on top of the Java Shared Data
Toolkits (JSDT) (www.javasoft.com) Application Programming Interfaces (APIs). A user
( a student or an instructor) can initialize and start a new session by sending a request to
the session manager which is running on the session server. Whenever a session is
started, other users can log in or leave that session freely. Each user is treated equally in a
session as one Session Client (SC). Through the SS, each SC can send messages to all
Server Internet Client
Session Session Session
Client Client Client
Figure 6. Relations between Session Server and Session clients and the peer-to-peer structure among
4.2.2 Java Media Frame (JMF)
JMF is the platform that we used to develop video players for both ICP and SVP.
A rich collection of APIs are provided by JMF for video playback, video capturing, and
video conference (www.javasoft.com/jmf). JMF supports most standard video formats,
such as Cinepak, MPEG, H.261, H.263 and JPEG. For streaming video application, RTP
is supported in JMF, which facilitates our implementation of transmitting, receiving, and
processing Multicast video media. The version of APIs we used for ISEE development is
4.2.3 Peer-to-peer Multicast
Multicast is used for the transmission of video streams. In the current version of ISEE,
only one-way multicast is implemented. Students can log into a session (group), and after
authorization begin receiving multicast streams from the live class or from the instructor.
As opposed to traditional client/server architecture, the instructor is also a common
member for a given session. Thus it is straightforward to support multiple channels for
multicast from other students. This is the so-called peer-to-peer multicast architecture.
Live Class ICEE
Figure 7. Multicast of Video Streams Supported in ISEE. Solid lines show the current streaming
directions implemented in ISEE. Dotted lines show peer-to-peer multicast architecture
4.2.2 ISEE communication protocol
Each message sent to and received from the SS is encoded and decoded based on
a communication protocol we defined for the ISEE. This protocol covers five types of
information that are transmitted across SS and SCs. They include: comment messages
issued from a user (session client), the status information ( an user joins/leaves the
session), the hyperlink/URL (to be shared across session members), the font color for
messages sent from a SC ( to display messages on the chat room), and the video
timestamp from the mediator SC (so the SC can synchronize with each other). After
being decoded, these messages can be utilized by the SC that receives it to trigger
corresponding actions (table 1). For example, if the messages received by a SC contain a
chat message, it will be presented on the ICR of ISEE; if a timestamp is identified, it will
be presented on the ICR with a link that can update the IVP. The font information
received will decide the color of the message presented on the ICR (to help distinguish
different participants). The URL information will provide a hyperlink on the ICR.
The following pair of tags are used in the ISEE communication protocol:
<Message> … </Message> Text message from users
<TextColor> … </TextColor> Color of the text from users
<Status> … </Status> The status information for users
<URL> … </URL> URL to be shared
<SynVideo> … </SynVideo> The timestamp for synchronization
Table 1: Tags used in the ISEE Protocol for Information Sharing
5. Innovations in the ISEE
The ISEE provides several innovations.
5.1 Private/Shared binary learning space
Studying in a conventional live classroom, students share common learning
processes: they see the same examples, listen to the same lecture from the instructor, join
the same discussion, and look at the same slides provided by the instructor. However,
students usually cannot control the pace of the class: they cannot slow it down when
coming across problems, or speed it up for the familiar content. The same problem exists
in most collaborative CDL systems.
For most collaborative CDL systems, a shared video stream (real-time or pre-
recorded ) is available for students to share the learning process. But either the students
are not allowed to control the playback of the video stream, or they are reluctant exercise
control (e.g. stop or rewind) because they usually do not want to interrupt other members’
watching, especially when the students are not familiar with each other . And in such
systems, any operations on the streaming video will lead the video player to re-buffer,
which is time consuming, especially for slow network connections.
But students may still prefer to have video control functions (e.g. stop/resume,
jump backward/forward). In order to test our simple hypothesis, an informal survey was
In our survey, we walked around the campus of the University of North Carolina at
Chapel Hill and interviewed 20 students with the following question :
Imaging you are a member of a virtual class with 200 students and you are now
watching a video synchronously with others. You can discuss the content of the video
with others through a chat room. Do you think the control functions (like pause/resume,
jump backward/forward) are necessary and helpful to your study if you can always
synchronize with others at will? How about if you can not synchronize with others after
you exercise control?
18 of the 20 students said the control functions are necessary and helpful. One
said she did not know and one said he did not care. If no synchronizing function is
available, 13 of the 20 students answered they needed the control functions. One student
said he should be able to play back the video segments if he missed something important.
Clearly, this was not a controlled user study, but this informal assessment suggests that
students do want to have control over video they use in learning.
However, video control and synchronization are conflicting operations and most
of the collaborative distance learning systems which support synchronizing video do not
provide functions to let users fully control the playback of the “shared” video. ISEE
provides a shared/private binary mode mechanism that not only provides a private
version of the video player, which allows the students to fully control the playback of the
video, but also provides a synchronizing function (synchronize button) that allows the
students to synchronize their video player with other students with just a single click.
Students can easily shift between the two modes. Any operations to control the video
player will switch from shared mode to the private mode. A single press on the
synchronizing button will automatically bring it back to the shared mode.
5.2 2D video navigation
For the distance learning activity, there are two basic “dimensions” involved: a
temporal dimension and a spatial dimension. In the video playback, the temporal
dimension is defined as the process from the point when the video starts to the point
when the video ends. For example, real-time live video streaming is a one-way linear
process in the temporal dimension. The student is unable to stop, rewind, or jump around
in this dimension. The temporal dimension is definded by the duration of the video.
For distributive collaborative learning, students are arrayed in various locations.
We define this as the spatial dimension of distance learning. The spatial dimension is
determined by the current active number of students in a specific study session. Thus the
spatial dimension will change when students join or leave the session. Learning
individually has no variation on the spatial dimension.
Both the temporal and spatial dimensions exist in most CDL systems. However,
in most cases students can only work in one dimension. For example, students are able to
download a video clip and jump around in the temporal dimension individually, but
unable to synchronize with other students. Or students can join a collaborative system
and share information across group members who are located in the spatial dimension but
can not control the shared video player in the temporal dimension. Students cannot jump
backward or forward in the streaming video.
ISEE is designed to allow students be able to navigate in both the temporal and
spatial dimensions. We call it a 2D distance learning system. For a pre-cached video in
the IVP, a scroll bar is provided (center of Fig.1) to allow students determine what to see
next in the temporal dimension. A single click on the “Synchronize” button will
immediately re-synchronize the student’s IVP with the mediator’s, which is also
synchronized by most group members. With the Interactive Chat Room (ICR), students
can exchange messages with other group members located remotely in the spatial
dimension. Timestamps following the username of each message make it quite
straightforward to track the timestamp of the message sender’s IVP.
5.3 Smart Alignment
In ISEE, services are provided for students to receive information from multiple
channels: real-time video streaming channel, shared browser channel, pre-cached video
channel, and shared chat room channel. A mechanism called “smart alignment” is
provided in ISEE that enables students to automatically align one channel with another.
For example, a single click on the timestamp followed by the sender’s name in the chat
panel will align your IVP with the sender’s IVP. A double click on the hyperlink in the
chat panel will update the URL of the shared browser. Pressing the “synchronize” button
will align your IVP with the mediator’s. Inputting an URL in the text field beneath the
SWB will “broadcast” that URL to all the group members and immediately update all the
6. Pilot user study and future work
6.1 pilot user study
A pilot test of the ISEE was conducted on April 2, 2002 in a class in the School of
Information and Library Science, University of North Carolina at Chapel Hill. There
were 37 students enrolled in this class and 8 students volunteered to test the ISEE. Except
for a web page to illustrate how to download and install the ISEE on their laptops, we did
not provide any training on how to use ISEE. The subjects (students) had ISEE installed
in their own laptops before the class and connected with the class through the wireless
Ethernet. The wireless network is Multicast enabled. The class lasted for about 1 hour
and 15 minutes. After the class started, subjects were asked to go to places outside the
classroom with their laptop and to login as a virtual student. The live class was captured
by a IBM ThinkPad UltraPort Camera which was mounted on a laptop and the live
streaming video was sent out using Multicast. The Instructor also logged in to mediate
and answer questions.
A video clip, Senses and Sensitivity 1, was downloaded to the subjects’ laptops
before class. During the class, the instructor opened the ISEE user interface and displayed
it through the project. All the students in the classroom could see the operations of the
instructor as he used ISEE. Fifteen minutes before the class was over, subjects were
invited to return to the classroom for a brief discussion about the ISEE. Finally, after the
class, several students and the instructor were interviewed. The results for the class
discussion and the interviews are summarized as follows:
From the students’ point of view:
1. Students were excited about the tools. With no extra equipment, a very simple
installation and no special training, they could see the live video of the class
from other places.
2. The audio channel is very important. They could accept low quality video, but
the audio needed to be high quality so they could feel they are in the class. They
complained that sometimes they could not hear clearly from the live classroom
stream even though the video was clear enough.
3. They suggested that any actions issued from the instructor should be “broadcast”
to students. For example, when the instructor paused the video playback and
began to discuss it, the remote students do not realize it and their IVP still played
4. There were still some bugs in the system. For example, the chat room sometimes
failed to refresh the screen freezes for a while until a new message is received or
From the instructor’s point of view:
1. Even with a carefully planned presentation, there was substantial overload. The
instructor had difficulty reading the remote chat while lecturing and leading
2. There is need for an assistant. Having someone control the live classroom
camera is important so that the camera can capture whomever is talking rather
than being fixed on one scene. Likewise, having an assistant read and respond to
the chat is necessary during lecture/presentations and helpful during class
From the saved logs of the chat room we found that most of the conversation was
about how to use the system instead of focusing on the content of the video and class. For
example, except the hellos from each user, discussions about how to control the two
sound channels ( from video clips and from live class) and how to mute one of them,
about how to synchronize the video and complaints about the sound and excitement upon
finding new functions were the dominant comments in the chat room.
An earlier version of ISEE has been demonstrated in the NCNI student fellowship
presentation conference (RTP,2002). The demo was manipulated by the first author who
developed the system. Feedback was very positive with great interest in Smart
Alignment, 2D navigation and the private/shared view functions demonstrated by ISEE.
Suggestions included further development of peer-to-peer multicast and interface design.
Another demo will be held at that Internet2 Spring 2002 meeting in Alexandria.
6.3 Future work:
Before we say that ISEE provides some helpful functions for Collaborative
Distance Learning, formal user studies are needed to evaluate the user (instructor and
students) acceptance of the technology and the effects of the technology on learning. We
are planning various user studies next: lab tests, interviews, surveys, classroom testing for
a whole semester, and the applications of ISEE in other settings like discussions and
Other work includes fixing the existing bugs, equipping some high quality
capturing device for audio, adding shared whiteboard, providing real-time image (frame)
grabbing, presenting a thumbnail of frames in the chat room and extending the peer-to-
This work was supported by grants from the North Carolina Networking Initiative
(www.ncni.net) and NSF grant number IIS 0099638. Thanks are also due to instructors
and students who have observed and used ISEE.
1. R.H Zakon, “Hobbes’s Internet Imeline” http://info.isoc.org/guest/zakon/Internet/
2. Gibbons,J.F., Kincheloe, W.R., and Down, K.S.(1977). Tutored videotape
instruction: a new use of electronics media in education. Science. 195:1139-1146.
3. Stone,H.R.(1990). Economic development and technology transfer: Implications
for video-based distance education. In M.G.Moore(Ed.), Contemporary issues in
American distance education(pp231-242). Oxford, England: Pergamon.
4. Sipusic, M., Pannoni, R., Smith, R., Dutra,J., Gibbons., J.and Sutherland, W.
(1999). Virtual Collaborative learning: A Comparision between Face-to-Face
tutored Video Instruction and Distributed Tutored Video Instruction(DTVI). Sun
Microsystems Laboratories, Inc. TR-99-72.
5. Smith, R., Sipusic, M., and Pannoni, R. (1999). Experiments Comparing Face-to-
Face with Virtual Collaborative Learning. Sun Microsystems Laboratories, Inc.
6. D.Harris and A.DiPaolo, “Advancing asynchronous distance education using high
–speed networks” IEEE Trans.Education. vol 39, pp.444-449, Aug 1996.
7. B.Peyser, which streaming media tool do I need?
8. community tolls evaluation:
9. Maly,K., Abdel-Wahab,H., Overstreet,C.M., Wild,C., Gupta,A., Youssef,A.,
Stoica,E., and Al-share,E. 1997. Distance learning and training over intranets.
IEEE Internet Comput. 1,1(May/June), 60-71.
10. Maly,K., Abdel-Wahab,H., Wild,C., Overstreet,C.M., Gupta,A., Abdel-hamid,A.,
Ghanem,S., Gonzalez,A. , and Zhu,X. 2001. ACM Journal of Educational
Resources in Computing, Vol. 1,No.1.
11. J.-N. Hwang, S.G.Deshpande, and M.-T. Sun, “A virtual classroom for real-time
interactive distance learning” , Proc. ISCAS,vol.3, June 1998, pp.611-614.
12. JJ Cadiz, Anand Balachandran,Elizabeth Sanocki, Annop Gupta, Jonathan
Grudin,Gavin Jancke. Distance Learning Through Distributed Collaborative
Video Viewing. 2000. MSR-TR-2000-42.
13. H.P. Dommel, J.J. Garcia-Luna-Aceves, Floor control for multimedia
conferencing and collaboration. Mulitmedia Systems, vol 5, 23-38, 1997