On-Demand Video Tagging, Annotation, and Segmentation in Lecture Recordings t...
Burleson
1. Submitted for special issue of Journal of Engineering Education (D. Budny, ed.)
Educational Innovations in Multimedia Systems
1
Wayne Burleson, Aura Ganz, Ian Harris
Department of Electrical and Computer Engineering
University of Massachusetts Amherst, MA 01003
{burleson, ganz, harris}@ecs.umass.edu
1
An early version of this paper won the Ben Dasher Best Paper Award at the 1999 Frontiers in Education Conference [
FIE99]
Abstract: Multimedia systems have emerged as one of the fastest growing segments of computing systems and thus need to
be well integrated into a computer engineering curriculum. Fortunately the teaching and learning of multimedia systems
can be aided with novel instructional techniques based on multimedia. The Multimedia Curriculum project at the
University of Massachusetts Amherst is developing a unified set of instructional materials on the engineering techniques
used in the design and test of hardware, software and networks for multimedia. This large project includes three facets: 1)
multimedia instructional modules using web-linked Digital Video Disks, 2) multimedia communication utilities to facilitate
student interaction and 3) multimedia component design projects. In this paper, we explain our approach to using
multimedia as both content and instructional technology and briefly present preliminary results in each of the three
facets.
1.0 Why Multimedia Systems?
We define a multimedia system to be a computer-based communications system which delivers heterogeneous and
compressed/coded/encrypted content (text, audio, video, graphics) from a source or storage device and transfers it over a
heterogeneous channel (Internet, wireless network, local area network) to an end-user while maintaining perceptual
integrity (Figure 1).
New multimedia systems have emerged in many forms in the last 5 years and are now a major driver in the design of
computer hardware, networks, and both system and application software. Processors, RAM, cache, disk, display, sound
card, graphics card, network card, operating system, browser and editors have all been modified to target multimedia
systems. Multimedia presents a new class of applications in computing which is quite different than the business and
scientific applications that drove previous generations of computing systems. It spans real-time computing, signal
processing, and communications issues and thus requires a very wide range of technical background. Multimedia systems
engineering is an opportunity to substantially update and invigorate undergraduate computer engineering curricula while
providing exciting new content for exploring new instructional methods and technology.
2. Submitted for special issue of Journal of Engineering Education (D. Budny, ed.)
Figure 1: Multimedia Systems: The User's View
Multimedia systems also provide a motivating theme for integrating many of the fields of computer engineering thus
encouraging multidisciplinary work.
• First, multimedia systems require a systems approach to design that covers the generation, transmission, storage and
retrieval of widely varying content. Algorithm, hardware and software design problems can be unified in an integrated
context, thus providing students with a ``big-picture" view of computer engineering {DeMan}. We use this systems
approach in design projects at all levels, requiring students to work in teams and deal with many design issues and
constraints simultaneously.
• Secondly, multimedia systems designs involve a significant amount of statistical and probabilistic analysis for the
estimation of performance and signal quality, thus substantially motivating math and signal processing courses in the
curriculum. Simulation and visualization tools are used to show how system and algorithmic choices impact the
resulting media product and result in variable run-times.
• Third, multimedia systems provide very tangible functionality (and misfunctionality!) to students, hopefully
motivating them and showing real applications without compromising engineering fundamentals.
2.0 The UMASS Multimedia Curriculum project:
“Using Multimedia to Learn Multimedia”
The UMASS Multimedia Curriculum project is led by 7 faculty in the Computer Systems Engineering area within the
Department of Electrical and Computer Engineering. Together, we are developing a set of integrated instructional tools
and curricular innovations which are unified by the common theme of multimedia systems. Our approaches to this
problem consist of:
• multimedia instructional modules using web-linked Digital Video Disks described in Section 3,
• multimedia communication utilities to facilitate student interaction described in Section 4, and
• multimedia component design projects described in Section 5.
Extensive multimedia archives, source code, demonstrations, modules, authoring materials and project details can be found
at the Multimedia Curriculum project web-site:
http://www.ecs.umass.edu/ece/dvd
3.0 Multimedia Instructional Modules
We are developing interactive multimedia instructional modules in the following 6 areas related to multimedia systems.
• MODULE 1: Natural Content Coding (video, audio)
3. Submitted for special issue of Journal of Engineering Education (D. Budny, ed.)
• MODULE 2: Synthetic Content Coding (graphics)
• MODULE 3: Multimedia Networks
• MODULE 4: Multimedia Architectures and Operating Systems
• MODULE 5: Design/CAD for Multimedia Hardware
• MODULE 6: Testing of Multimedia Systems
Together, the modules provide an integrated problem domain which cuts a wide swath through the field of computer
engineering. With 7 different faculty developing 6 different modules in this project, we are able to amortize the module
development infrastructure over a wide range of content and teaching styles. We believe that the overhead required to
develop any re-usable instructional modules is significant enough that the infrastructure should be used by a variety of
faculty in a variety of courses. We have developed a generic module template with utilities for novel functions (interactive
video lectures, interactive video demonstrations, links to related design projects) , but we see significant variation in the
individual modules due to the different topic areas and teaching styles.
These modules directly impact 6 undergraduate ECE courses as well as being used as short course tutorials for distance
learning and self-paced instruction. The modules are based on a combination of interactive Digital Video Disks and Web
technology, allowing a combination of high quality video and audio along with hot-links to the Web. The next three
sections show three different approaches to multimedia modules production. The first, WebDVD, provides multiple
camera angles and instructional materials but in an unsynchronized format. The philosophy here is to allow asynchronous
use of the materials with synchronization left up to the viewer. This is similar to reading a textbook or set of notes that go
along with a lecture . The second format, MANIC, provides streamed audio over the Internet with synchronized HTML
slides. MANIC uses RealPlayer and various features of the Netscape and Internet Explorer web browsers to maintain
synchronization and allow indexing. The third format is a low-cost CD/DVD format that requires only very basic PC
equipment and a digital video camera and microphone. It is intended for faculty that do not have access to a professional
video studio or video-serving capability.
3.1 WebDVD: Multiple Camera Angles with Unsynchronized Instructional Materials
Figure 2a Clip from WebDVD course on VLSI testing for video coding modules (Prof. I. Harris)
3.2 MANIC: Streaming Audio with Synchronized HTML
Figure 2 shows a screen shot from a module on VLSI design which illustrates many of the concepts in our multimedia
modules. The video is captured in the professional studios of our Video Instructional Program just like any regular
UMASS academic course or short course. We then digitize the video and integrate it with other course materials onto the
DVD. Initially we use the MANIC system, developed in the UMASS Computer Science department {MANIC}, to
integrate slides and video stills with synchronized audio into an Internet-based format that can be viewed by a standard
4. Submitted for special issue of Journal of Engineering Education (D. Budny, ed.)
browser (e.g. Explorer or Netscape) and media player (e.g. RealPlayer). The authoring process is quite efficient
compared to tools like Macromedia Director, taking only about 2-3 hours for every hour of live lecture. This ratio depends
on the amount of animation and annotation that is desired. UMASS undergrad Brendan English converted 31 hours of
video lectures containing 528 slides to the MANIC format during a winter vacation week.
The MANIC system allows easy navigation through the modules, either asynchronously by choosing the slides from a table
of contents or search engine, or synchronously by just playing the audio and allowing the slides to be advanced
automatically. At any point, the student can stop the presentation and interact. Current forms of interaction include:
1) navigation through slides via a hierarchical PDF-style table-of-contents,
2) following links to other slides, glossaries and external web-sites,
3) searching for key words that appear in the slides titles and text (search for keywords in video, audio or image is not
currently supported).
Details on MANIC can be found at: http://manic.cs.umass.edu/
Figure 2b: Clip from VLSI course using MANIC format (Prof. W. Burleson)
3.3 Low-Cost Video Production Environment
For faculty that do not have access to a professional video studio, we have also experimented with a low cost multimedia
production method that incorporates web based material, slides, video and audio. The required hardware and software can
be purchased for less than $5000.
Hardware:1. Pentium PC 500MHz, 256 MB RAM, 2. Hauppauge WinTV (Video capture card that transfers in real time the
video signal from a digital video camera to the computer screen). 3. Hitachi digital video camera, 4. Microphone
Software:1. Microsoft Media Player,2. HyperCam ( Video screen capture software), 3. Adobe Premier (Video editing)
The digital video camera captures the instructor that faces the PC screen that displays: 1) the web browser that views the
slides prepared in html format and 2) the video of the instructor that is captured by the video capture card. The HyperCam
software captures in real time all the contents of this screen including the mouse movements and highlighting and records
it in .avi format (10 MB per minute of video clip). The .avi file is cut and trimmed in Adobe Premier and and then
compressed to an MPEG format. Although this format is intended for DVD/CD rather than Internet streaming, clips can be
viewed at: http://dvd1.ecs.umass.edu/miu/. A sample of the format can be seen in Figure 2c. This work was developed
with Annan Phonphoem and Kitti Wongthavarawat
5. Submitted for special issue of Journal of Engineering Education (D. Budny, ed.)
Figure 2c Low-cost video instructional format used on Networking module (Prof. A. Ganz)
3.4 Encouraging Student Interaction with Modules using Applets, Quizzes and Demonstrations
Students can also interact with the lectures by requesting examples and demonstrations, which are written either as Java
Applets run on the client or CGI programs run on the server. We have developed a number of Java applets that provide
students with an interactive multimedia experience that enables them to grasp a number of networking concepts.
1. Routing concepts. The user can input any network layout (any number of nodes, links and link weights). Students can
program their own routing algorithm and compare it to the classical shortest path.
http://dvd1.ecs.umass.edu/kenny/j_spf.htm. The complex dynamics of ad-hoc networking are shown in Figure 3a.
2. ATM routing algorithms. The user can input the network layout (nodes, links, links' weight, capacity and length), the
routing algorithm (min-hop and min-delay) and the percentage of video traffic in the network. Figure 3b displays the
delay versus virtual circuit (VC) arrival rate and also the percentage of accepted VCs versus the VC arrival rate.
http://dvd1.ecs.umass.edu/qyu/ece696/index.html
3. Weighted Fair Queue Service Discipline: The applet shows how the weighted fair queuing discipline works in a
network switch under different prioritized multimedia traffic loads. Figure 4 shows this applet. The applet can be
viewed and downloaded at http://dvd1.ecs.umass.edu/miu/JavaApplets/WeightedFairQueue/Wfq.htm
Additional applets can be viewed at http://dvd1.ecs.umass.edu/miu/.
Figure 3a: Applet for observing ATM performance on video traffic
6. Submitted for special issue of Journal of Engineering Education (D. Budny, ed.)
Figure 3b Applet demonstrating ad-hoc networking both visually and statistically. User enters parameters.
Figure 4 Java applet demonstrating Weighted Fair Queueing of multimedia traffic both visually and statistically.
User enters rate and priority parameters.
An example of an interactive demonstration of the MPEG encoding algorithm is shown in Figure 5. In the first screen
shot, the student chooses a number of parameters for the encoding algorithm and then submits them to the MPEG coder. In
the second screen shot, the results of the student’s parameters choices are shown, including the statistics of the MPEG-1
encoding program (run-time, compression ratio, file sizes, etc.) as well as the actual test video (in this case, the benchmark
7. Submitted for special issue of Journal of Engineering Education (D. Budny, ed.)
Table Tennis video which shows varying object motion (ball and paddle) and zooming). This work was done with
UMASS PhD student Jeongseon Euh and uses the public domain MPEG coder developed at UC Berkeley.
Figure 5: Interactive MPEG demo
Another form of student interaction is an on-line homework system. At any point during a module or outside of the
module, students can choose a set of questions specified by topic and difficulty level. For example, after viewing 15
minutes of a lecture on Video coding algorithms, the student might request 3 intermediate-level questions on motion
estimation techniques or, 2 questions drawn from the last n minutes of the presentation or m slides of the presentation.
Another scenario is that a well-prepared student who already has some background in a particular area might request some
questions before viewing the lecture. This allows students to make the best use of their time by “testing-out” of sections
that they may already know. The homework can also be used to preview lectures and review material for exams, projects,
job interviews, etc. Homework problems can also link to the interactive demos and applets thus providing continuity
between the lectures and the homework. The software provides immediate feedback after the students submit answers. The
feedback includes the score for each session, accumulated score for all sessions, how long students take to finish each
session, right or wrong for each question (if an answer is wrong the program will present a correct answer to students).
Details and a demo of the homework system can be found at http://tikva.ecs.umass.edu/quiz/index.html. The homework
system was developed with Jianxin Wang, Xin Liu and Noel Llopis. In the questions/answers database we are developing
the ability to display multimedia files such as figures, video and audio. Figure x shows an example where students choose
one of several hyperlinks to MPEG video files as a response to a multiple choice question.
Our homework system is different than the many other on-line homework systems (see recent FIE proceedings) in the
following ways:
• emphasis on multimedia in question and answers
• integration with modules (questions tagged by topic, time and difficulty)
8. Submitted for special issue of Journal of Engineering Education (D. Budny, ed.)
• mixed-mode to support automated grading as well as human grading
• integrated with applets and demos
3.5 Handwritten Presentation Styles
In order for any teaching technology to be quickly adopted in a widespread manner, it is essential that additional effort on
the part of the teacher should be minimized. Even the most well-intentioned teacher is unlikely to have sufficient free time
in his/her schedule to learn a complicated new presentation format. For this reason, our work strives to allow the teacher to
present educational material in an intuitive fashion allowing informal handwritten “back-of-the-napkin” formats. The
challenge is to balance the intuitiveness of the presentation style with the ease of digitizing the educational material for
distribution to students.
Figure 6: Annotation of an electronic document using a digitizing pad and Microsoft Paint
One method which we have investigated to provide a natural presentation interface is the use of a drawing tablet, together
with a simple drawing tool such as Microsoft Paint which is shipped with Windows98. The Cross IpenPro which we are
using, allows the mouse to be controlled by moving a pen across a pad. A mouse click is performed by applying downward
pressure to the pen. By using the drawing pad with a drawing program, the teacher can draw directly onto the computer
screen, and annotate an existing drawing, as shown in Figure 6. We have found this interface to be a nice addition to
electronic presentation materials (i.e. Powerpoint) because it allows a very natural interface for the teacher, while also
instantly digitizing the data for distribution either on CD/DVD or on the Internet.
Another approach we use to allow annotation of electronic materials is a video overlay technique. This requires video
mixing capability which we have in our video studios. Unlike the IpenPro, this results in a video rather than a still image
overlay, allowing full animation to capture the comments and annotation style of the instructor. Figure 7 below shows the
video overlay technique in a VLSI chip design course where the instructor is tracing a schematic over a representation of
the manufacturing masks.
9. Submitted for special issue of Journal of Engineering Education (D. Budny, ed.)
Figure 7 Video Overlay of Powerpoint slides originally from Digital Integrated Circuits by J. Rabaey, Prentice-Hall
4.0 Multimedia Communication Utilities
We have developed a variety of multimedia utilities to support communication and collaboration in conjuction with the
instructional modules of Section 3 and the design projects of Section 5. Despite the promise of technology for the formal
delivery of educational materials, we feel that some of the strongest benefits of technology are just to enable informal
human-to-human interaction. Ideally each student could freely and frequently interact with the professor, the teaching
assistant and their fellow students. However, the reality of time limitations and scheduling conflicts makes this difficult.
One of the simplest, most general and most effective solutions has been the use of electronic mail and bulletin boards. We
have developed several utilities that make it easier to discuss, critique, annotate, and revise multimedia objects. These are
used in standard academic courses but are most important in design project courses.
One of the most fundamental utilities allows the student to submit an arbitrary multimedia file into the on-line
homework system. Thus, a student can provide an audio file or a graphics file as a solution to a homework or quiz
problem. This significantly expands the homework system beyond the limitations of automated multiple-choice question
styles and also supports students with various communication and learning disabilities.
A multimedia bulletin board allows students to post their work and then view and comment on it. This is similar to
students preparing their own Web pages except that commenting and interaction between students is better supported.
Contributions can be listed either threaded by subject, or ordered by time of posting. Multimedia files such as figures,
video and voice can be posted.
We have developed a multimedia whiteboard that allows students and the professor to actually collaborate on the design
of a multimedia document. A document can be viewed with annotations in color and text added by the professor and
students (Figure 8). A distributed client-server framework allows multiple documents from different courses and projects
to be open at the same time, encouraging multidisciplinary collaboration. In addition to the whiteboard, we have also
developed a multimedia chat program and a multimedia forum which are based on the same principles but have
different modes of interaction. For example, chat and forum store a transcript of the interaction while whiteboard just
stores the most current document. Chat is synchronous while the whiteboard and the forum are optionally asynchronous.
In conclusion, all of these Internet-based communication systems can benefit from specific multimedia support. This
work was done by UMASS undergraduate students Jeff Peden and Chris Leonardo and was presented to a multimedia
research community at [ICME 2000] and an educational community at [EWME 2000].
10. Submitted for special issue of Journal of Engineering Education (D. Budny, ed.)
Figure 8: Multimedia Whiteboard showing on-line commenting by multiple designers of a VLSI layout for Huffman
coding
5.0 Multimedia Component Design Projects
Multimedia modules provide a good introduction to a topic area, but a more effective and realistic technique for learning
computer engineering involves a team-based hands-on design project. To support this, we have developed an archive of
hardware and software design projects related to multimedia systems and linked to the instructional modules at various
levels. To facilitate these projects, we rely on the modules described in Section 3 as well as the utilities in Section 4. We
have incorporated multimedia themes into several of our undergraduate design project courses. Projects vary from software
and networking to architecture, microprocessor and VLSI design depending on the course. In the VLSI design course, we
have projects in Audio Coding, Discrete Cosine and Wavelet Transforms, Image Filtering, LZ coding, Arithmetic coding,
Huffman Coding, Cryptography and Video Motion Estimation. Although these are components or sub-systems, rather
than complete multimedia systems, they all present significant challenges for the senior-level students since they have to
thoroughly understand the correct functionality of the blocks before they can design the circuits and generate proper test
vectors for simulations.
We have found that the modules are very useful as a review for students doing multimedia design projects. Students in a
VLSI project course are designing a Huffman coding unit that uses a barrel shifter and a RAM. By searching for these
keywords in the VLSI module, they can get a quick overview of the design issues for these blocks. Even for students that
have never studied a particular topic, the modules can be used to give a quick introduction. For example, we have students
designing a Lempel-Ziv compression chip who have never encountered the basic Lempel-Ziv algorithm. Rather than give
them a journal article or textbook as background, or ask them to just “find it on the Web", or require a semester length
Data compression course, the module allows a quick introduction to the topic from an instructor whose presentation style
and reputation they already know.
Expertise in the testing of multimedia systems is becoming required for system designers in industry and should therefore
be introduced at the undergraduate level. Multimedia systems can be used to teach many concepts of testing in a very
tangible way. Testing, in turn, forces students to carefully look at their design from an external viewpoint, addressing
issues such as I/O, standards, and manufacturing defects. Several aspects of multimedia systems make their testing a
unique problem:
11. Submitted for special issue of Journal of Engineering Education (D. Budny, ed.)
• Real-Time Systems: Multimedia systems are typified by a high bandwidth user interface which must meet real-time
constraints. The high bandwidth required in many applications, such as video, can push design performance limits and
necessitate delay optimization and delay verification {Sifakis92}.
• Heterogeneous Systems: A typical multimedia application is built from many heterogeneous components which are
required to interface the digital representation of multimedia data with the physical senses of the user, typically
through audio and video. System components vary widely in terms of data throughput and implementation technology.
Hierarchical design is well-known but a hierarchical testing {HierMurr90} approach is required to accomodate the
various testing approaches required for each component.
• Mixed-Signal Systems: The large majority of multimedia systems are based on digital components. However, because
human senses are inherently analog, multimedia systems must include analog components as well. Mixed-signal
design and testing present challenges beyond either digital or analog {MilorMixed98}.
• Standards Implementation: Intense consumer demand for multimedia systems has driven standardization efforts in
order to increase volume and reduce costs. The purpose of standardization is to create a common high-level
functionality across different products, while giving a large degree of freedom in system implementation. Because
different implementations of a standard will vary greatly, standards compliance tests {MpegComp95} must be
designed which are independent of internal design.
Each design project is associated with a set of objectives which the students must achieve. The goals fall into three
categories which collectively span the entire flow of a project from concept to product.
• Design - Students must design a circuit, or some subcomponents of a circuit, from a high-level specification. The
abstraction level of the design created varies by project. For example, a project for a sophomore class in logic design
may involve board-level design using gate-level components, while a senior VLSI design project will involve layout-
level design.
• Verification -Students must verify the correctness of their design. This involves the definition of a simulation model or
a design. The simulation model will either be in an HDL language (VHDL, Verilog) or in a software language (C,
Java), as appropriate to the design level. Verification will also require that the students design a set of functional
vectors which stimulate potential design errors. The process of creating a set of test vectors forces the student to have a
thorough understanding of the internals of the design.
• Manufacturing Test - Students must verify that the physical circuit performs the task specified by the design. Notice
that this step is different from verification because manufacturing test identifies physical faults in the manufacturing
process, as opposed to design errors. The nature of the manufacturing faults is dependent on the level of the
manufacturing process. Our projects involve three different manufacturing processes: board-level, FPGA, and LSI
fabrication. Requiring students to test for manufacturing faults provides them with an understanding of the
manufacturing process, as well as the design process.
We now present three undergraduate design projects that involve typical multimedia testing and verification issues. The
scope of these projects has been limited to fit a typical junior or senior level lab course. By fully understanding the design
and test of one of these designs, students get a glimpse of the design and test of multimedia systems in general.
Project 1: Bouncing Lights Design The circuit designed in this project use light emitting diodes (LEDs) to represent a
bouncing ball whose movement is controlled by a set of switches and a potentiometer. The output interface of the final
circuit is a row of LEDs, all of which are lit except one. The unlit LED acts as the ``ball'' which ``bounces'' between the left
end and the right end of the row of LEDs. The inputs to the circuit are a reset switch which starts the bouncing, a hold
switch which holds the position of the ball, and speed potentiometer to control the speed of bouncing.
This is intended to be an early, sophomore level, student design experience, involving breadboarding using primarily TTL
logic components. The circuit is a finite state machine with a variable speed clock to control the bouncing speed. The circuit
12. Submitted for special issue of Journal of Engineering Education (D. Budny, ed.)
design is subdivided into two components, (1) finite state machine logic to sequence the lights, and (2) clock generation
logic to generate the variable speed clock. The finite state machine logic is based on an up/down counter used to select the
unlit LED. The clock control logic uses an analog-to-digital converter to translate the potentiometer resistance into the
downsampling rate for the clock.
This project fulfills our educational objectrives because it exposes students to many fundamental aspects of multimedia
design early in their undergraduate education. This design includes a simple video interface in the form of LEDs, and
enables students to explore issues relating to subjective qualities of the output. For example, a common problem is that an
LED may appear dim because it is alternating between on/off states more quickly than is perceptible. The student goals of
this project are twofold, (1) design and build the circuit, and (2) test the circuit for manufacturing defects. Since this project
involves manual wiring, many defects related to wiring are to be expected. For the purpose of simplicity, we require the
detection of only {em open} defects which occur when a wire between two points has not been connected. As an additional
challenge, the students are asked to diagnose the faults as well as possible without probing internal signals, using only the
input and output interface. Requiring students to diagnose without probing gives the students an appreciation of the
difficulty of test observability, particularly in VLSI chip design where internal probing is often impossible.
Project 2: VLSI Design of Modular Exponentiation for Cryptography The Montgomery modular multiplication
algorithm is a central component of many cryptography systems. Since cryptography is important for a large class of
Internet-based secure multimedia applications, this design is a component in many multimedia systems. Since
encryption/decryption are performed on the incoming/outgoing media data streams, modular exponentiation is part of the
critical performance path and requires delay verification. Modular exponentiation is also part of the RSA encryption
standard. Although the implementation of the exponentiator is not defined in the RSA standard, the behavior of the
exponentiator must be correct in order to guarantee the compliance of the larger RSA system.
As an implementation model, we use the linear array design presented in [Paar]. This project is intended for a senior
level lab course, and assumes a basic understanding of VHDL. The student objectives of this project are (1) design the
component in VHDL, (2) generate several test streams for validation, and (3) insert test hardware into the design and
explore the area/testability tradeoff. We have made a VHDL simulation model if a length 4 linear array version of this
design, as well as an associated test bench, which should be used as a reference model. The complexity of the project can be
scaled by the instructor by varying the length of the linear array of processing elements, or by targeting an FPGA
implementation rather than an ASIC implementation. Evaluation of the area/testability tradeoff is performed by modifying
the design by inserting test registers. The test registers improve testability while sacrificing area. The testability of the
modified designs is evaluated using tools which we have developed here, as well as freeware for test generation and fault
simulation [PROOFS].
Project 3: Video Motion Estimation Array Motion estimation is critical part of the popular MPEG video compression
standard, used in a wide range of multimedia equipment. Motion estimation is the most compute-intensive step of the
MPEG encoding task. The high performance standards of digital video applications make delay verification essential in
motion estimation. The verification of this design necessitates adherence to the requirements of the MPEG standard, and
gives students exposure to compliance testing.
As an implementation model, we use the 16 element linear array design presented in [Konstantinides] which estimates the
motion of a 16x16 pixel reference block within a 32x32 pixel search window. Each processing element of the array shown
in Figure 9 compares the reference block to a unique 16 pixel wide column of the search window. The student objectives of
this project are (1) design the component in VHDL, (2) generate several test streams for validation, and (3) insert test
hardware into the design and explore the area/testability tradeoff. The complexity of the project can be scaled by altering the
search window size to reduce the number of processing elements, or by altering the number of bits used to represent each
pixel. We have developed a VHDL model for the design which can be used for verification. The verification problem here
is interesting because the results of a design error can be observed visually by using the component to encode a video
sequence. Students can perform rudimentary diagnosis relating visual effects on the encoded image to design and
manufacturing defects.
13. Submitted for special issue of Journal of Engineering Education (D. Budny, ed.)
Figure 9 Motion Estimation Array for Video Coding (Project 3)
6.0 Evaluation
This project is ongoing and presents numerous innovations that must be evaluated to determine their value in improved
teaching and learning. We have just begun this large effort by piloting the various modules, utilities and projects into our
own curriculum. After these pilot runs, we will also actively disseminate the materials to five other diverse institutions
around the world (ENST Paris, Pusan Nat. Univ., Korea, National Technological Univ., Smith College, and the Springfield
Technical Community College). Our evaluation strategy consists of four components:
1) Student evaluation of the teaching materials. Throughout the design of the modules, the utilities and the design
projects we have incorporated numerous ideas from students. Undergraduates are heavily involved in the project. Early
survey of the Networking module indicated 98% student satisfaction. A recent graduate course in Wireless
Communications indicated that the pace of our modules was too slow and that the video lectures were too low quality to
provide an stimulating learning environment. We are addressing this concern by using better video coders and trying to
more carefully target the modules to different student audiences.
2) Assessment of student learning using pre- and post-tests for students using the modules versus a control group. Our
homework system administration tools provide the course instructor the ability to edit and modify quiz files, and analyze
quiz results. A more informal approach will be used for student projects due to the difficulty in formally assessing learning
in design.
3) Feedback from the 5 dissemination institutions on the benefits of the modules, utilities and projects to students and
faculty. The authoring tools and source code are made available to these institutions as well as a module explaining how
to develop your own modules.
4) Feedback from industrial partners who hire our students and use our courses for continuing education (e.g. Compaq,
Sun, Samsung, Intel, Cadence, Cisco, Motorola.)
References:
[DeMan] H. DeMan, Keynote address in ACM/IEEE Design Automation Conference, 1997.
[FIE99] W. Burleson, A. Ganz, I. Harris, "Educational Innovations in Multimedia Systems", Frontiers in Education
Conference '99
14. Submitted for special issue of Journal of Engineering Education (D. Budny, ed.)
[ICME00] W. Burleson, J. Peden, C. Leonardo, "Distributed VLSI Design Education using The Multimedia Online
Collaboration Architecture (MOCA)", Proc. of European Workshop on Microelectronics Design, (EWME), Kluwer, May
2000
[EWME00] J. Peden, W. Burleson, C. Leonardo, "The Multimedia Online Collaboration Architecture: Tools to Enable
Distance Learning", Proc of IEEE International Conference on Multimedia, August, 2000
[MANIC] M. Stern, J. Steinberg, H.I. Lee, J. Padhye, J. Kurose, "MANIC: Multimedia Asynchronous Networked
Individualized Courseware," Proc. of Educational Multimedia and Hypermedia, 1997.
[MpegComp95], P. Meehan and N. Hurst and M. Isnardi and P. Shah, MPEG Compliance Bitstream Design, International
Conference on Consumer Electronics, June, 1995, pp.174-175.
[MilorMixed98],L. S. Milor, A Tutorial Introduction to Research on Analog and Mixed-Signal Circuit Testing, IEEE
Transactions on Circuits and Systems II: Analog and Digital Signal Processing, vol. 45, no. 10, 1998.
[Sifakis92] J. Sifakis, Real-Time Systems Specification and Verification, International Symposium on System Synthesis,
September,1995.
[HierMurr90], B. T. Murray and J. P. Hayes, Hierarchical Test Generation Using Precomputed Tests for Modules, IEEE
Transactions on Computer Aided Design, vol. 9, no.6, June, 1990, pp.=594-603.
[Paar] T.Blum and C. Paar., Montgomery modular exponentiation on reconfigurable hardware. Proc. of IEEE Symposium
on Computer Arithmetic, April 1999.
[PROOFS] T. M. Niermann, W.-T.Cheng, and J.~H. Patel. Proofs: A fast, memory-efficient sequential circuit fault
simulator. IEEE Transactions on Computer-Aided Design, 11(2):198--207, February 1992.
[Konstantinides] V. Baskaran and K. Konstantinides, Image and Video Cokmpression Standards, Kluwer Academic
Publishers, 1997.
Acknowledgements:
The UMASS Multimedia Curriculum project is funded by the National Science Foundation grant EIA-9812589 and the
University of Massachusetts. We acknowledge the work, support and ideas of all members of the project, including
Professors M.Ciesielski, I.Koren, C.M.Krishna and F.S. Hill; UMASS students: J. Peden, B. English, J. Euh, A.
Nalamalpu, C. Duggirala, J.Wang, X. Liu, N. Llopis, K. Zheng, A. Phonphoem and K. Wongthavarawat.We also thank
the Video Instructional Program, the Engineering Computer Services and the UMASS Computer Science Department
MANIC project for providing technical services and use of equipment.