1. Interactive Content Authoring for A/153
ATSC Mobile Digital Television
Employing Open Source Tools
Brad Fortner – Ryerson University
bfortner@ryerson.ca
ATSC TechZone Pavilion
Booth # N4039

2. interactive content authoring?
Lack of books or
information on
subject that
provide
examples and
clarification.

3. evolution

4. ATSC M/H content group
Brad Fortner Tyler Pham Gregory Kon Steven James May
From Left To Right: Many Ayromlou, Victor Nguyen, Alireza
Tahmasbi, Raymond Padillo, Jeremy Littler

5. a decade later

6. a decade later- standards
A/153 part 5 set standards for the rich media environment for ATSC mobile dtv

7. a decade later- content
Steve Bratt W3C Technical Plenary 2003
"The Semantic Web: common framework that allows data to be shared and reused
across application, enterprise, and community boundaries."

8. a decade later- MPEG 4
MPEG-4 multimedia object based coding standard
Today: way more than a video compression format
H.264/MPEG-4 Part 10

9. a decade later- MPEG 4
MPEG-4 now 28 parts from which the principles of the
rich media environment for A/153 was derived

10. a decade later- learned?
=
Rich Media Application
Linked Data
Multimedia Objects
+

11. agenda
Could you
provide some
background?
• background
• the three parts to rich media delivery
• A/153’s rich media environment
• what and why multimedia object coding is important
• A/153’s “push in advance of consumption” content model
• multimedia scenes
• presentation format
• SVG Tiny 1.2
• ECMAScript
• packaging format
• DIMS and carousels
• open source software available for prototype development
• pitfalls and presentation takeaways

12. rich media delivery
Let’s do a
traffic app!
presentation
format
packaging
format
transportation
format
Rich Media
Content
Server(s)
Browser
with Rich
Media
Engine
Let’s do a
traffic app!
Browser
with Rich
Media
Engine

13. rich media environment
Let’s do a
traffic app!
presentation
format
packaging
format
transportation
format
Browser
with Rich
Media
Engine
SVG Tiny 1.2
ECMA Script
Dynamic
Interactive
Multimedia
Scenes (DIMS)
A/153
Let’s do a
traffic app!
Browser
with Rich
Media
Engine

14. Scene Stream Server
scenes in MPEG-4
Server A
Server B
How’s the Weather
App progressing?
Server C
primitive multimedia objects
How’s the
Weather App
progressing?
rendered scene on browser

15. Scene Stream Server
scenes in MPEG-4
Server A
Server B
How’s the Weather
App progressing?
Server C
primitive multimedia objects
How’s the
Weather App
progressing?
rendered scene on browserscene description

16. scene description MPEG-4
primitive multimedia objects
How’s the
Weather App
progressing?
rendered scene on browserscene description
Environmental
Objects
Office Desk Chair
Businessman
Objects
Businessman Speech Bubble
Fragment Fragment Fragment Fragment Fragment Fragment Fragment Fragment
Hierarchy

17. scene description MPEG-4
primitive multimedia objects
How’s the
Weather App
progressing?
rendered scene on browserscene description
• places (arranges) media objects on interface
• transforms media objects attributes to alter appearance
o scale, transparency, color, animation etc.
• provides timing between media objects and streamed data
• provides for user interactivity
• called (microDOM) uDOM in SVG Tiny 1.2
+ =

18. MPEG-4 object coding
primitive multimedia objects
How’s the
Weather App
progressing?
rendered scene on browserscene description
+ =
+ =
primitive multimedia objects rendered scene on browser
scene description

19. presentation
format
packaging
format
transportation
format
SVG Tiny 1.2
ECMA Script
Dynamic Interactive
Multimedia Scenes
(DIMS)
A/153
pushing multimedia objects

20. push content model
• return path not guaranteed in ATSC M/H
• return path
• device dependent
• network availability dependent
• forward, store, use content design model
• applications must be designed accordingly
“push content in advance of consumption model”

21. push content model
“push content in advance of consumption model”
• six “Multimedia Scenes” are used in the weather app
• they are transmitted all at one time
• a small number of SVG fragments can be reused over a number of scenes
• judicious use of SVG fragments reduce transmission requirements

22. push content model
“push content in advance of consumption model”
• ECMAScript is used to program the interactive buttons to switch between multimedia scenes

23. presentation format
So how does one
create these multimedia
objects?

24. SVG tiny 1.2
=
• XML-based file format used for describing two-dimensional vector graphics
• editable text
• human-readable, machine-readable
• can be searched, indexed, and scripted
raindrop object (fragment) markup in SVG Tiny 1.2

25. SVG tiny 1.2
But what about my
cute cat pictures?
JPEG and PNG
encoded to
base64

26. SVG tiny 1.2
3X size: above rendered at 1262 * 727 – originally designed for 416 * 240
• traffic object is originally a 293 * 220 bitmap
• does not scale with other SVG objects in scene

27. SVG tiny 1.2 scene creation

28. scene description
SVG simultaneously creates multimedia objects
and scene description as one authors content
SVG Link to raindrop
fragment
Raindrop fragment screen positioning and
sizing markup. This is used as
scene description.

29. SVG tiny 1.2 scene creation
SVG Tiny 1.2 rich
media application
displayed in Opera
browser
.svg

30. freely available software
Show Me
The Free!!!
• any text editor will edit SVG and XSLT
• integrated developer environment (IDE)

31. freely available software
Show Me
The Free!!!
• SVG vector graphics editors
“VISIO like” “Illustrator like”online

32. freely available software
Show Me
The Free!!!
• SVG vector graphics libraries

33. freely available software
SVG Tiny 1.2 rich media
application displayed in Opera
browser
inkscape svg object
text inside svg
markup
icon svg objects from
openclipart.org
objects chosen for display
on client using ECMA
script
weather from rss feed
inserted using XSLT
inkscape svg object
text inside svg
markup
event handler buttons are
ECMAScript functions

34. software used weather app

35. packaging format
Scene Stream
Server
scene
description
Server A
Server B
MPEG-4 Terminal
Server C

36. 3GGP DIMS object packaging
Server A
Server B
Server C
Scene Stream
Server and
DIMS Packager
packaged
scene
description
and objects
highly constrained
network and
devices

37. open source DIMS software
What! Open
Source DIMS
software. WOW!
“as is”

38. open source DIMS software
Understood.
GPAC is useful
for prototypes.
• GPAC Project on Advanced Content
• implementation of the MPEG-4 systems standard
• written in ANSI C (cross platform)
• tools for media playback, vector graphics, MPEG-4 authoring and distribution
• consists of
o multimedia packager called MP4Box
o some server functions (experimental)
o multimedia player called Osmo4
• MP4Box
o generates 3GPP DIMS packaging from SVG files
• Osmo4
o displays SVG Tiny 1.2 scenes packaged in 3GPP DIMS files

39. 3GGP DIMS carousel types
object
0
object
1
object
2
object
3
object
0
object
1
object
2
object
3
static carousel0
1
23
4
GPAC Experimental DIMS
Stream Server
• static carousel
• repeatedly sends scene description and objects
• repeats because devices tune in at different times

40. 3GGP DIMS carousel types
• dynamic carousel
• sends updates to screen elements
0
1
23
4
GPAC Experimental DIMS
Stream Server
dynamic carousel (update)
object
0
object
1
object
2
object
3
object
4
object
0
object
1
object
2
object
3
object
4
object
5
object
5
changechangechange

41. GPAC DIMS static operation
object
0
object
1
object
2
object
3
object
0
object
1
object
2
object
3
static carousel0
1
23
4
GPAC Experimental DIMS
Stream Server
GPAC DIMS server command-line interface instructions are as follows
GPAC
Software
Packager
Destination
IP Address
Time
Between
Carousel
Sends
Initiates
DIMS
Packaging
Name Of File
To Be Packaged
And Sent
Streams
Scene
Live

42. GPAC OSMOS4 player
object
0
object
1
object
2
object
3
object
0
object
1
object
2
object
3
static carousel0
1
23
4
GPAC Experimental DIMS
Stream Server
GPAC OSMOS4 player command-line interface instructions are as follows
GPAC
OSMOS4
Player
Looks for Session
Description
Protocol from
DIMS server

43. GPAC dynamic updates
Primary Stream
GPAC Experimental DIMS
Stream Server
0
1
23
4

44. Primary Stream
GPAC Experimental DIMS
Stream Server
0
1
23
4
GPAC dynamic updates
Update Stream
Primary Stream
GPAC Experimental DIMS
Stream Server
0
1
23
4

45. GPAC dynamic updates
Update
Main Carousel0
1
23
4
GPAC Experimental DIMS
Stream Server
Linked Data ServerLinked Data Server
Update

46. GPAC dynamic updates
Update
Main Carousel0
1
23
4
GPAC Experimental DIMS
Stream Server
Linked Data ServerLinked Data Server
Update

47. GPAC dynamic updates
Linked Data Server
• Linux BASH shell script to collect linked data and write the update
file (doScript)
• BASH: wget --output-document=camera.jpg http://www.toronto.ca/trafficimages/loc9.jpg
• BASH/DIMS: a="<Replace ref="trafficImage" attributeName="xlink:href"
value="data:image/ jpeg;base64,"$(base64 camera.jpg)"" />"
• BASH: echo $a > updateFile.svg

48. Update Stream
Primary Stream
GPAC Experimental DIMS
Stream Server With BASH
Script, Update File, XSLT
Program and Traffic
Prototype SVG File
0
1
23
4
GPAC
Software
Packager
Destination
IP Address
Time
Between
Carousel
Sends
Initiates
DIMS
Packaging
Name Of File
To Be Packaged
And Sent
Name Of Update
File That GPAC
Watches
Streams
Scene
Live
GPAC dynamic operation

49. freely available software
Show Me
The Free!!!

50. Pitfalls – SVG tiny 1.2 viewer
GPAC OSMO4
Browser Render
(Packaged DIMS File)
OPERA
Browser Render
(SVG File)
Lack of a fully
implemented
freely available
SVG tiny 1.2
standard viewer

51. Update Stream
Primary Stream
GPAC Experimental DIMS
Stream Server With BASH
Script, Update File, XSLT
Program and Traffic
Prototype SVG File
0
1
23
4
GPAC – experimental nature
• dynamic carousel
• Prototype crashes when primary and update streams sent too
close together

52. takeaways - moving objects
presentation
format
packaging
format
transportation
format
SVG Tiny 1.2
ECMA Script
Dynamic Interactive
Multimedia Scenes
(DIMS)
A/153

53. takeaways - push
• return path not guaranteed in ATSC M/H
• return path
• device dependent
• network availability dependent
• forward, store, use content design model
• applications must be designed accordingly
“push content in advance of consumption model”

54. SVG simultaneously creates
multimedia objects and scene
description as one authors
content
Takeaways - SVG

55. takeaways - carousels
Primary Stream
GPAC Experimental DIMS
Stream Server
0
1
23
4
Update Stream
GPAC Experimental DIMS
Stream Server
0
1
23
4

56. free prototype software

57. more information
www.openmobiledtv.org

58. Brad Fortner – Ryerson University
bfortner@ryerson.ca
ATSC TechZone Pavilion
Booth # N4039
www.openmobiledtv.org
Ryerson University Booth
North Hall - #N524
International Research Park