research questionshow can (Twitter) content contributed around large-scale broadcast news events inform journalistic inquiry?what kinds of insights and analyses are enabled through visual analytic tools?how to build better tools to support journalistic inquiry?
“Hey Fellas,have a look at thisscoop I justpulled from TWITTER.”!
designing for journalismwhat would a journalist ask of Twitter?tendency to cover “decisive moments”, divergent points of view, audience criticism/applausenewsworthiness: favors surprising, unusual, good / baduseful indicators: – sentiment: applause, criticism, controversy, good / bad – response magnitude: quotes, topics / issues – unique / novel contributions
research questionshow can we transform the consumption of video content created by users attending large-scale events?what kinds of practices and user tasks can be enabled through a new experience?how to build better tools to leverage the social context of video capture?
Lyndon Kennedy and Mor Naaman. Less Talk, More Rock (WWW 2009)
challenge: evaluate!produce insights about use and usefulness that go beyond a speciﬁc implementation
challenge: evaluate!what are the diﬀerent factors that are in play when users interact with the new application?
challenge: evaluate!how do users perceive and understand the presentation and interaction mode?
challenge: evaluate!how does the new experience help (or hinder) users in performing tasks?
two-pronged approachdirectly evaluate the output of the algorithms (quantitative)deep, extensive evaluation of the users interaction with the system or interactive prototype (qualitative) in the context of their taskread more: Olsen (UIST ’07); Naaman (MTAP ’10)
Multiplayer study goalswhat utility is provided to event content consumption?how users interact with a multi-perspective videos from an event?
laboratory studytwo tasks/scenarios: – seen a show, reminisce/share – considering a show, checking it outdata collected – think-aloud data – detailed, open-ended questionnaire – Interaction logging
data analysisthemes extracted from notes, transcriptslogging data examined for support
insights: event viewingseeing all activities on stage “There’s a lot of activities on stage ... normally I would not be able to see everything at once and this allows me to do that”
insights: event viewing“being there” “get an idea of what it’d be like to be in the audience” “This preview is better than a seating chart!”
insights: interactionthumbnails natural, timeline not so muchaudio switching problematic
insights: taking controlusers expressed need, desire to generate new media artifacts If I really like a certain performance then I’d set it up as a mini-editor to have a super-customized version of this great event”
Vox study goalshow do journalists use Vox Civitas and what is its journalistic utility?what features do journalists interact with?what kinds of insights does Vox Civitas support?
online studydeployment scenario/task:: – “use the tool to develop two story pitches for the event”data collected – detailed, open-ended questionnaires – interface loggingparticipants – 15 / 18 with training in journalism
data analysisthemes extracted from answers to questionnairetypes of story angleslogging data examined for support
insights: utilityideas for follow-up; mechanism to ﬁnd sources “ﬁgure out what statement resonated the most with the public” “used as a compliment to stories that include more sound data”
insights: story anglesaudience response in multiple scales – individual – aggregate volume, sentiment
insight: using searchKeyword searches and ﬁltering helped ﬁnd story angles “Further investigation into statistics on college loan debt. How much are students carrying? And how long does it take to pay oﬀ?”
insight: using searchKeyword searches and ﬁltering helped ﬁnd story angles ““I chose the keyword ‘overseas’. This gave me more of mixed emotions for the audience due to the slash in tax breaks being given to companies who ship their jobs overseas”
summary created new experiences for social media content from/ about events evaluations resulted in insights that are not necessarily tied to particular implementation
thanks to… joint work with superstars: Nicholas Diakopoulos Funda Kivran-Swaine Tayebeh Yazdani Zihao Yu
thank you firstname.lastname@example.org mornaaman.com @informor join us at rutgers SMIL (email me or introduce yourself)