Interactive Content Authoring for A/153 ATSC Mobile Digital Television Employing Open Source Tools - Brad Fortner, Ryerson University, NAB 2012 Presentation

1. 1

2. • NAB 2010 came looking for information on how to
author interactive content for the medium
• Other than standards papers there are few to no
authoritative sources on how to use these
standards in this environment.
2

3. • Interactive Broadcast Learning Lab
• Digital Hydra --- TransMedia
• 2M Funding 2yr Interactive Broadcast Ontario
• Joined ATSC
• Hosted Canadian ATSC Rollout Conference
• Tested datacast ideas using ATSC from signage to
robotic control applications
3

4. • ATSC M/H content group
4

5. • Mobile interactive television 10 years later
5

6. • A/153 Mobile Standard Adopted
• A/153 Part 5 Application Framework
• Standard methodology for multimedia creation
• Brings interactivity to TV medium
• Puts standardized technology “in the band” or
“under the hood” to provide for interactivity
• Significant enough to build upon for future
advances in transmission such as ATSC 2.0 and
3.0
• Just the start of “under the hood” technologies
that have the potential to provide tremendous
opportunity to the medium
6

7. • 2003 W3C Plenary Session
• Initial Web --- Tomorrows Web
• XML, SOAP, SVG, XHTML common
• Metadata Universe
7

8. • TV industry knows MPEG-4 as a video/audio
compression format
• Way more that a compression format.
• Multimedia object based coding format
8

9. • MPEG-4 now best thought of as a multimedia
object coding standard
• Way more that a compression format.
• 28 Parts MPEG-4 standard
• Cross into areas shown in slide
• MPEG-4 now 28 parts from which the
principles of the rich media environment
for A/153 was derived
9

10. • Nutshell - Most important things learned
• How to work with linked data
• How to work with Multimedia objects
• How to mold them into an A/153 Part 5 Rich
Media Application
10

11. • Agenda
11

12. • First
• Presentation Format
• Part that one really thinks of as the content
• Second
• Packaging Format
• Prepares content for distribution
• Dependent on transportation format
• Third
• Transportation Format
• Medium of distribution
12

13. • A/153 Part 5
• Based on Open Mobile Alliance Rich Media Environment
• OMA-RME
• Presentation Format
• W3C SVG Tiny 1.2
• ECMAScript
• Mobile Profile
• known as Javascript
• Packaging Format
• 3rd GenerationPartnership Project (3GPP)
• Dynamic Interactive Multimedia Scenes (DIMS)
• Transportation Format
• ATSC Mobile DTV Format
• A/153
• Uses ATSC transmitters to deliver content
• A/153 Part 5
• Umbrella standard
• Contains elements
• Of application creation
• Of Delivery
• Of Control
• Our work
• Prototype discovery driven
• Involves presentation and packaging formats only
• Did not include transmitting applications
• No access to transmitter technology
13

14. • Classic academic paper or book on MPEG-4
objects
• Primitive multimedia objects are arranged
and rendered into a multimedia scene
• MPEG-4 scenes were designed to pull primitive
objects from any server on a network
• A scene stream server was devised to identify
scene elements
14

15. • The scene stream server is responsible to deliver
the Scene Description for a Rich Media Application
that defines the scene layout
15

16. • The scene description can define each object,
element or fragment that makes up a primitive
multimedia object.
• The terms object and element are often used
interchangeably when describing a scene of
object.
16

17. • Scene Description
• advises browser of scene objects
• places (arranges) media objects on interface
• transforms media objects attributes to alter
appearance
o scale, transparency, color, animation etc.
• provides timing between media objects and
streamed data
• provides for user interactivity
• called (microDOM) uDOM in SVG Tiny 1.2
17

18. • Classic explanation of arranging primitive
multimedia objects into a rendered scene on a
browser never made much sense to me
• However when the primitive multimedia objects
become traffic images, weather icons and
information, news headlines, alerts and
advertising this workflow starts making more
practical sense for Rich Media Applications
18

19. • So in a nutshell ATSC A/153 Part 5 is all about
creating and pushing multimedia objects through
the Rich Media environment prescribed by the
standard.
19

20. • push content model
• push content in advance of consumption
model
• return path not guaranteed in ATSC M/H
• return path
• device dependent (out of band)
• network availability dependent
• forward, store, use content design model
• applications must be designed accordingly
20

21. • Weather Application
• six “Multimedia Scenes” are used in the
weather app
• they are transmitted all at one time
• a small number of SVG fragments can be
reused over a number of scenes
• judicious use of SVG fragments reduce
transmission requirements
21

22. • In this application ECMAScript is used to program
the interactive buttons to switch between
multimedia scenes
• SVG provides viewports and the buttons are
programed to display 416*240 pixel slices
22

23. • In A/153 Part 5 W3C’s SVG Tiny 1.2 is used to
create multimedia objects
23

24. • XML is the basis for metadata
• XML-based file format is text
• editable text
• human-readable, machine-readable
• can be searched, indexed, and scripted
• XML markup is shown here in the creation of a
raindrop object
• Note the use of the <g> </g> tags.
• These are used to group a series of
elements
• In this case they are used to identify the
elements that makeup the raindrop object
24

25. • JPEGS and PNG’s are binary based and are
encoded into base 64 when placed in an SVG file.
• XML is comprised of text only
• XML must be readable by people
• Example of a separate, specific or individual
encoding type specified by the MPEG 4 Object
Coding standard.
25

26. • SVG Vector Graphics Scale Well
• 3X size: above rendered at 1262 * 727 – originally
designed for 416 * 240
• traffic object is originally a 293 * 220 bitmap
• Bitmap does not scale with other SVG objects in
scene
• SVG scales well into HDTV and beyond
26

27. • Multimedia scene creation is a three step project
• First define and group objects
• Secondly position them within the scene
27

28. • Scene Description
• SVG simultaneously creates multimedia objects
and scene description as one authors content
28

29. • Third (if required) use XSLT (eXtensible Stylesheet
Language Transform) to include desired data from
linked data sources.
• Accomplished by adding appropriate eXtensible
stylesheet language to the original SVG document.
• Appropriate objects will be added to the
document and the XSLT processor will output a
completed .svg file ready for the packaging format
29

30. • Freely available software for presentation format
• any text editor will edit SVG and XSLT
• integrated developer environment (IDE)
• Eclipse is most evolved
30

31. • SVG editors
31

32. • SVG Open Libraries
32

33. • Objects and their open source/open standards
33

34. • software used in weather app
34

35. • Originally MPEG-4 provided Scene Description and
objects were pulled from multiple network
locations
• Packaging came along for mobile devices that had
limited on constrained network connections
35

36. • DIMS server for highly constrained networks and
devices.
36

37. • Open source DIMS software
• GPAC Project on Advanced Content
• implementation of the MPEG-4 systems standard
• Not broadcast product, distributed “as is”
• Useful for prototyping only
37

38. • consists of
• multimedia packager called MP4Box
• some server functions (experimental)
• multimedia player called Osmo4
• MP4Box
• does 3GPP DIMS packaging from SVG files
• Osmo4
• displays SVG Tiny 1.2 scenes packaged in
3GPP DIMS files
38

39. • DIMS is all about carousels
• Two types
• Static and Dynamic
• static carousel
• repeatedly sends scene description and
objects
• repeats because devices tune in at different
times
39

40. • dynamic carousel
• sends updates to screen elements
40

41. • GPAC DIMS static carousel operation
• Command line interface instructions for DIMS
Live streaming
• DIMS generates an IP stream that A/153
transports to player
41

42. • GPAC OMOS 4 player
• Command line interface instructions for
starting OSMOS4
• Opens IP stream that GPAC Dims server generates
42

43. • Traffic App uses the Static carousel to send the
basic scene description only
43

44. • Traffic App then uses the Dynamic Carousel to
update screen by sending all objects
44

45. • To send updates GPAC watches a text file
• When new information is written to the text file
GPAC immediately sends its contents as an update
45

46. • To send updates GPAC watches a text file
• When new information is written to the text file
GPAC immediately sends its contents as an update
46

47. • Linux BASH shell script to collect linked data and write
the update file
• BASH: wget --output-document=camera.jpg
http://www.toronto.ca/trafficimages/loc9.jpg
• This line uses a BASH command that saves
the next traffic image to be sent
• BASH/DIMS: a="<Replace ref="trafficImage"
attributeName="xlink:href"
value="data:image/ jpeg;base64,"$(base64
camera.jpg)"" />“
• This uses a DIMS replace command and
initiates the base64 encoding built into Linux
and saves it as a string
• BASH: echo $a > updateFile.svg
• This BASH line writes the string as a text file
to the update file. GPAC then sends the
command.
47

48. • GPAC DIMS dynamic carousel operation
• Command line interface instructions for DIMS
Live streaming
• Name of update file added to command line
that initiates dynamic updates
• DIMS generates an IP stream that A/153
transports to player
48

49. • Freely available software used for packaging
format
49

50. • No freely available fully implemented SVG Tiny 1.2
standard viewer
• Makes one wonder what will find their way into
A/153 Part 5 compatible devices
50

51. • Experimental nature of GPAC
• dynamic carousel
• Prototype crashes when primary and
update streams sent too close together
51

52. • So in a nutshell ATSC A/153 Part 5 is all about
creating and pushing multimedia objects through
the Rich Media environment prescribed by the
standard.
52

53. • push content model
• push content in advance of consumption
model
• return path not guaranteed in ATSC M/H
• return path
• device dependent (out of band)
• network availability dependent
• forward, store, use content design model
• applications must be designed accordingly
53

54. • Scene Description
• SVG simultaneously creates multimedia objects
and scene description as one authors content
54

55. • Traffic App uses the Static carousel to send the
basic scene description only
• Traffic App then uses the Dynamic Carousel to
update screen by sending all objects
55

56. • free software used in prototype environment
56

57. • more information www.openmobiledtv.org
57

58. 58