Hypermediated TV

Hypermediated

TV

Ryan Shaw unmediated.org UC Berkeley SIMS Yahoo! Research Berkeley

Context , not just content

Early attempts at interactive TV and video focused on content

But the contexts surrounding media are a better place to interact

Which are we preserving when we talk about archiving TV?

Remix culture as a collective filter

"Video search... will only work if we have millions of people informing our collective knowledge of what is worth our individual attention, and that can only happen if we can annotate, share, and remix video ."

John Battelle, 4/27/05

Commons-based peer production

Modularity

Production can occur incrementally and asynchronously

Granularity

Contributors can make very small commitments

Low-cost integration

Robust systems for quality control

Collectively tagging photos

A socio-technical system

Technological (networks, software)and social (people) components

Design must be informed by an understanding of:

how people interact with each other

how people interact with media

how people interact with technologies

Connect metadata producers and consumers

Media Streams Metadata Exchange

Distributed production of metadata

Fine-grained contributions, integrated incrementally

Allow anyone to annotate any media quickly and easily

Associate annotations with concepts to allow disambiguation

milkshake brings all the boys to the yard, and they're like, it's better Daniel “ I'm looking for an angelic female character , slowly looking up" Takuya "How about Ah! My Goddess, second season , episode 5 ?"

shot requests shot pointers music videos remix requests (ignored)

Shot Archive shot pointers & descriptions shot requests

shot pointers Shot Archive shot pointers & descriptions

music videos requests (granted!) Shot Archive Editing Rule Engine

Only store pointers to content

Metadata is associated with pointers to temporal and spatial regions of multimedia content

Remixers produce edit decision lists , not edited copies of content

Has the Family Movie Act legalized this approach to remixing copyrighted content?

In conclusion

It is still too difficult to find and author media

Context metadata can solve this, if it can be produced on a large scale

We can design tools to mediate a social process around media

Fans' interactions with these tools will produce context metadata

Thanks! [email_address]

Commons-based peer production Operating Systems Encyclopedias

AnimeMusicVideos.Org

323,000 members

59,000 AMVs by 20,000 creators

7,500-8,000 unique visitors per day

Very active forums for discussing anime and AMVs

Studied anime fans to inform our design

Catalyze creative ideas for innovative tools

Use cases for prioritizing design concepts

Working model of users , domains and contexts of interaction

User-centered design process

In-depth interviews with remixers

Online survey of 125 fans

Media metadata experiment

User personas

Scenarios of how tools might be used

Daniel: the editor

" I like to have something brand new every single video... [It's] a creative outlet for the things I want to do.”

Takuya: the expert

" I'm a trivia freak, so it just completely makes me ecstatic if I know something somebody else doesn't.”

Kira: the newbie

"hi, i'm new here and i've downloaded alot of AMV's in my time here.

unfortunately, i am unable to make any videos as of yet; but, i have an idea for one... i really really really want to see this one done! please, please!! "

"Cal"

My contributions

Reconceptualization of media search and media authoring

Detailed data on how a community describes, looks for, and remixes media content

Framework for understanding interactions with media in a particular domain

Next steps

Begin prototyping interfaces for MSMDX tools

Analyze queries for anime content from non-fans to see if metadata is useful outside community

Identify other communities that might use such tools (video bloggers, indie journalists)

An explosion of on-demand multimedia

But we lack tools to manage media

Multimedia content is still:

Hard to search for

Difficult to author with

People perceive characters, action, events Arthur Dent in his bathrobe , shortly before the Vogons destroy Earth

Computers perceive files, pixels, bits hhg-tlr3_m480.mov Quicktime 00:00:37:2

Media Streams (1995)

Analyzed shot requests

"I need at least one anime where the character is eating pancakes , preferably out at a diner ."

Object Setting Action

series "Cowboy Bebop"

cast "Miaka"

role "villain"

body "muscular"

object "pancakes"

number "three"

setting "enchanted forest"

action "eating"

genre "sci-fi"

mood "scary"

plot/theme "about a breakup"

production "loopable"

Asked fans to describe shots

Select temporal or spatial regions and add free-text annotations

Good match between requests and descriptions

Community-editable domain ontology

Links episodes to characters , character names to physical appearances , etc.

If someone labels a scene with "Vincent" associate that with "male, tall, long dark hair, tattoo"

Match up requests specifying body types with descriptions specifying characters

Closing points

anime fans collaborate to produce new works out of existing media

we can design tools to mediate and improve this collaboration

fans' interactions with these tools will produce semantic metadata

semantic metadata can make it easier to find and author media

Fans are active cultural producers

Already finding, describing, and remixing and sharing multimedia

Dedicated/obsessed enough to spend time contributing

Fandom is "serious leisure"

milkshake brings all the boys to the yard, and they're like, it's better milkshake brings all the boys to the yard, and they're like, it's better

milkshake brings all the boys to the yard, and they're like, it's better Music and lyrics provide the structure for the remix

Planning on paper

Lyrics and timings are written out with notes on what shots to use

Metadata for credibility and reputation

All data is linked to identity of its creator

System for implicitly and explicitly rating contributions

Allow building of social and cultural capital

MSMDX media player

"View source" for multimedia

A media player which allows jumping from remixed content to the original source from which it was taken

Viewers can bookmark and tag audio and video of interest: tags are saved to central database

MSMDX editor plugin

Use song lyrics or keywords to directly query for relevant scenes

Selected clips can be drawn from personal collection or downloaded on demand for editing

Labels that remixer adds to source content while working are uploaded to central database

MSMDX template builder

Load a digital music file to create a template with timed lyrics

Add high-level descriptions of scenes to be used

Resulting template can be filled with specific content by a remixer, or automatically

Proposed data model

( data model diagram goes here)

One platform dominates

One tool dominates

Several steps to create an AMV

Storyboarding or other planning

Ripping source anime to a hard drive

Finding and isolating desired clips

Synchronizing with music

Lip-syncing

Creating visual effects

Some steps are tedious

Lip-syncing

Other steps are enjoyable

Lip-syncing

Interviewed 3 creators

One pair interview, one solo interview, about one hour each

All male, early 20s

Interviewees were surprisingly reflective

Expressed great interest in the project and staying involved

Ran a web-based survey

Analyzed scene requests

"I need at least one anime where the character is eating pancakes , preferably out at a diner ."

“ I'm looking for characters who have been betrayed/burned by another character romantically ."

“ A clip of Tamahome with money and Miaka hitting him ."

“ I'm looking for characters who have been betrayed/burned by another character romantically ."

Theme/Plot

“ A clip of Tamahome with money and Miaka hitting him ."

Object Action Character

Using tags to find photos

Remix culture as a collective filter

"Video search... will only work if we have millions of people informing our collective knowledge of what is worth our individual attention, and that can only happen if we can annotate, share, and remix video ."