PolemicTweet
Video Annotation and Analysis
through tagged Tweets
Samuel Huron, Petra Isenberg, Jean Daniel Fekete
06/09/13...
.
05/09/2013 IFIP Interact 2013 – Polemic tweet
Petra
Isenberg
INRIA
@dr_pi
Raphael
Velt
IRI
@raphv
Yves Marie
Haussonne
IRI
@ymh_work
Authors and Contributors
Samuel
Hur...
06/09/13 IFIP Interact 2013 – Polemic tweet
At IRI we organize seminars
on subjects such as
philosophy, design, and
digita...
SITUATION
05/09/2013 IFIP Interact 2013 – Polemic tweet
05/09/2013 IFIP Interact 2013 – Polemic tweet
SITUATION
05/09/2013 IFIP Interact 2013 – Polemic tweet
Previous process
1. Videotape
the seminar
2. Manually
annotate and
tag
3. Pu...
06/09/13 IFIP Interact 2013 – Polemic tweet
PROBLEM
05/09/2013 IFIP Interact 2013 – Polemic tweet
05/09/2013 IFIP Interact 2013 – Polemic tweet
Problem
2. Annotating a
video manually
is boring and
time-consuming
1. Brows...
Inspiration: Microblogging activities
05/09/2013 IFIP Interact 2013 – Polemic tweet
Communication devices
Motivation
05/09/2013 IFIP Interact 2013 – Polemic tweet
3. Provide an easily
accessible system
from annotation to
video a...
A certain type of tag: Sentiments
06/09/13 IFIP Interact 2013 – Polemic tweet
Current approaches to extract
sentiments
Human analysis
Natural Language
Processing
Our need
  Fast
  Language agnostic
  Accurate
  Cheap
06/09/13 IFIP Interact 2013 – Polemic tweet
Why not ask the tweet autor ?
Cost Benefit
- Fast
- Language agnostic
- Accurate
- Cheap, cost is distributed
- Require us...
DESIGN
05/09/2013 IFIP Interact 2013 – Polemic tweet
Design Goals:
05/09/2013 IFIP Interact 2013 – Polemic tweet
1. To crowdsource
conference video
tagging with
structured
sen...
0. Service Design
1. A special syntax to
enhance tweets
4. A video player with a
visualization
2. A social protocol to
org...
05/09/2013 IFIP Interact 2013 – Polemic tweet
1. Syntax properties and description:
GreenAgreement++
RedDisagreement--
Yel...
2. Social Protocol
05/09/2013 IFIP Interact 2013 – Polemic tweet
A. Before the
talk: Instruct
the crowd.
B. During the tal...
Instruct the crowd
05/09/2013 IFIP Interact 2013 – Polemic tweet
  The network
access policy
  The Twitter
hashtag
  Th...
3. Backchannel: Real Time Visualization
3. Backchannel: Real Time Visualization
3. Backchannel: Real Time Visualization
3. Backchannel: Real Time Visualization
3. Backchannel: Real Time Visualization
3. Backchannel: Real Time Visualization
3. Backchannel: Real Time Visualization
4. Metadata Video Player
http://goo.gl/zyJYCZ
4. Metadata Video Player
http://goo.gl/zyJYCZ
4. Metadata Video Player
http://goo.gl/zyJYCZ
4. Metadata Video Player
http://goo.gl/zyJYCZ
DEPLOYMENT
05/09/2013 IFIP Interact 2013 – Polemic tweet
25 to
Attendees
per event
05/09/2013 IFIP Interact 2013 – Polemic tweet
27
Events
9088
Tweets
recorded
300 1012
Unique
Twi...
EVALUATION
05/09/2013 IFIP Interact 2013 – Polemic tweet
05/09/2013 IFIP Interact 2013 – Polemic tweet
Questions
1. Is the system
sufficiently
engaging?
3. What is the
impact of
p...
Data Collection
05/09/2013 IFIP Interact 2013 – Polemic tweet
Record tweets sent
during the events
Log the user
interactio...
RESULT
IFIP Interact 2013 – Polemic tweet05/09/2013
Question 1:
Is the system sufficiently engaging?
05/09/2013 IFIP Interact 2013 – Polemic tweet
0.2 to 7.94Tweets
per minut...
05/09/2013 IFIP Interact 2013 – Polemic tweet
Question 1:
Is the system sufficiently engaging?
•  Type of Audience. (size,...
Question 2:
Does the visual backchannel
incite people to add tags?
05/09/2013 IFIP Interact 2013 – Polemic tweet
05/09/2013 IFIP Interact 2013 – Polemic tweet
Question 3:
What is the impact of presenting tweets and
tags on a web video ...
05/09/2013 IFIP Interact 2013 – Polemic tweet
Question 3:
What is the impact of presenting tweets and
tags on a web video ...
05/09/2013 IFIP Interact 2013 – Polemic tweet
Question 3:
What is the impact of presenting tweets and
tags on a web video ...
CONCLUSION
05/09/2013 IFIP Interact 2013 – Polemic tweet
The Evaluation was challenging
05/09/2013 IFIP Interact 2013 – Polemic tweet
  Deployed in the wild
  Long period of tim...
Result!
1.The system
engaged the
audience to provide
data for
crowdsourced video
tagging.
2. The visual
backchannel incite...
Next steps
05/09/2013 IFIP Interact 2013 – Polemic tweet
As classroom
presentations.
Synchronous
web seminar.
Public debat...
Possible optimizations:
05/09/2013 IFIP Interact 2013 – Polemic tweet
Shared attention. Tag properties. Scalability, user
...
Subsequent work:
05/09/2013 IFIP Interact 2013 – Polemic tweet
http://goo.gl/c2iqg0
Bubble-TV: Live Visual
Feedback for So...
MORE INFOS:
+ open source
+ details
THANKS: To people who used the system and people who fund this work:
Thank you !
Conta...
THE END
05/09/2013 IFIP Interact 2013 – Polemic tweet
Survey Result
  96% (25) of respondents attended at least one
conference had already used twitter at conferences
( 7% onl...
Survey Result
In the survey 14 users reported to know
what the Tweet Timeline Visualization is.
05/09/2013 WIID: Who Is In...
Cost and benefit of tagging technics
06/09/13 IFIP Interact 2013 – Polemic tweet
- Time consuming
- Cost is proportional
-...
Upcoming SlideShare
Loading in …5
×

Polemic tweet - IFIP Interact 2013

873 views
789 views

Published on

We present PolemicTweet a system with an encompassing, economic, and engaging approach to video tagging and analysis. Annotating and tag- ging videos manually is a boring and time-consuming process. Yet, in the last couple of years the audiences of events—such as academic conferences—have begun to produce unexploited metadata in the form of micropost activities. With PolemicTweet we explore the use of tagged microposts for both video annotation and browsing aid. PolemicTweet is a system 1) to crowd source conference video tagging with structured sentiment metadata, 2) to engage audiences in a tagging process, and 3) to visualize these annotations for browsing and analyzing a video. We describe the the system and its components as well as the results from a one-year live deployment in 27 different events.

Published in: Technology, Education
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
873
On SlideShare
0
From Embeds
0
Number of Embeds
202
Actions
Shares
0
Downloads
3
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide

Polemic tweet - IFIP Interact 2013

  1. 1. PolemicTweet Video Annotation and Analysis through tagged Tweets Samuel Huron, Petra Isenberg, Jean Daniel Fekete 06/09/13 IFIP Interact 2013 – Polemic tweet
  2. 2. . 05/09/2013 IFIP Interact 2013 – Polemic tweet
  3. 3. Petra Isenberg INRIA @dr_pi Raphael Velt IRI @raphv Yves Marie Haussonne IRI @ymh_work Authors and Contributors Samuel Huron INRIA - IRI @cybunk Jean Daniel Fekete INRIA @jdfaviz
  4. 4. 06/09/13 IFIP Interact 2013 – Polemic tweet At IRI we organize seminars on subjects such as philosophy, design, and digital cultures. !
  5. 5. SITUATION 05/09/2013 IFIP Interact 2013 – Polemic tweet
  6. 6. 05/09/2013 IFIP Interact 2013 – Polemic tweet SITUATION
  7. 7. 05/09/2013 IFIP Interact 2013 – Polemic tweet Previous process 1. Videotape the seminar 2. Manually annotate and tag 3. Publish the video online and let user annotate
  8. 8. 06/09/13 IFIP Interact 2013 – Polemic tweet
  9. 9. PROBLEM 05/09/2013 IFIP Interact 2013 – Polemic tweet
  10. 10. 05/09/2013 IFIP Interact 2013 – Polemic tweet Problem 2. Annotating a video manually is boring and time-consuming 1. Browsing a video after a conference is difficult 3. Tagging and structuring annotations are even more boring
  11. 11. Inspiration: Microblogging activities 05/09/2013 IFIP Interact 2013 – Polemic tweet Communication devices
  12. 12. Motivation 05/09/2013 IFIP Interact 2013 – Polemic tweet 3. Provide an easily accessible system from annotation to video analysis. 2. Provide a visual backchannel to incite people to tag their tweets. 1. Engage the audience: crowdsource video annotation and tagging by tweet.
  13. 13. A certain type of tag: Sentiments 06/09/13 IFIP Interact 2013 – Polemic tweet
  14. 14. Current approaches to extract sentiments Human analysis Natural Language Processing
  15. 15. Our need   Fast   Language agnostic   Accurate   Cheap 06/09/13 IFIP Interact 2013 – Polemic tweet
  16. 16. Why not ask the tweet autor ? Cost Benefit - Fast - Language agnostic - Accurate - Cheap, cost is distributed - Require user effort - Low recall
  17. 17. DESIGN 05/09/2013 IFIP Interact 2013 – Polemic tweet
  18. 18. Design Goals: 05/09/2013 IFIP Interact 2013 – Polemic tweet 1. To crowdsource conference video tagging with structured sentiment metadata. 2. To engage audiences in the tagging process. 3. To visualize annotations for browsing and analyzing the video.
  19. 19. 0. Service Design 1. A special syntax to enhance tweets 4. A video player with a visualization 2. A social protocol to organize the event over time 3. A real time visualization to read and write tweets
  20. 20. 05/09/2013 IFIP Interact 2013 – Polemic tweet 1. Syntax properties and description: GreenAgreement++ RedDisagreement-- YellowReferences== BlueQuestions?? Tag Semantic Color   Simple   Brief   Neutral   Easy to parse   Expressive   Unambiguous Design Rational
  21. 21. 2. Social Protocol 05/09/2013 IFIP Interact 2013 – Polemic tweet A. Before the talk: Instruct the crowd. B. During the talk: Crowdsource the video tagging with a backchannel. C. After the talk: Publish on a web video player.
  22. 22. Instruct the crowd 05/09/2013 IFIP Interact 2013 – Polemic tweet   The network access policy   The Twitter hashtag   The website URL   The tags
  23. 23. 3. Backchannel: Real Time Visualization
  24. 24. 3. Backchannel: Real Time Visualization
  25. 25. 3. Backchannel: Real Time Visualization
  26. 26. 3. Backchannel: Real Time Visualization
  27. 27. 3. Backchannel: Real Time Visualization
  28. 28. 3. Backchannel: Real Time Visualization
  29. 29. 3. Backchannel: Real Time Visualization
  30. 30. 4. Metadata Video Player http://goo.gl/zyJYCZ
  31. 31. 4. Metadata Video Player http://goo.gl/zyJYCZ
  32. 32. 4. Metadata Video Player http://goo.gl/zyJYCZ
  33. 33. 4. Metadata Video Player http://goo.gl/zyJYCZ
  34. 34. DEPLOYMENT 05/09/2013 IFIP Interact 2013 – Polemic tweet
  35. 35. 25 to Attendees per event 05/09/2013 IFIP Interact 2013 – Polemic tweet 27 Events 9088 Tweets recorded 300 1012 Unique Twitter users Deployment
  36. 36. EVALUATION 05/09/2013 IFIP Interact 2013 – Polemic tweet
  37. 37. 05/09/2013 IFIP Interact 2013 – Polemic tweet Questions 1. Is the system sufficiently engaging? 3. What is the impact of presenting tweets and tags on a web video player? 2. Does the visual backchannel incite people to add tags?
  38. 38. Data Collection 05/09/2013 IFIP Interact 2013 – Polemic tweet Record tweets sent during the events Log the user interaction on the web video player Two web surveys, one for participants, one for organizers
  39. 39. RESULT IFIP Interact 2013 – Polemic tweet05/09/2013
  40. 40. Question 1: Is the system sufficiently engaging? 05/09/2013 IFIP Interact 2013 – Polemic tweet 0.2 to 7.94Tweets per minute 0.12 to 2.13 Tags per minute 40% Tweets using our syntax 35 to 45 % Of all individuals used our syntaxe
  41. 41. 05/09/2013 IFIP Interact 2013 – Polemic tweet Question 1: Is the system sufficiently engaging? •  Type of Audience. (size, culture, equipment, incentive, distribution) •  Type of Speaker and Twitter Reaction. •  Type of Content. •  Observance of the PT Protocol. Influential factors for engagement
  42. 42. Question 2: Does the visual backchannel incite people to add tags? 05/09/2013 IFIP Interact 2013 – Polemic tweet
  43. 43. 05/09/2013 IFIP Interact 2013 – Polemic tweet Question 3: What is the impact of presenting tweets and tags on a web video player?
  44. 44. 05/09/2013 IFIP Interact 2013 – Polemic tweet Question 3: What is the impact of presenting tweets and tags on a web video player? Visualization User seek activity ?
  45. 45. 05/09/2013 IFIP Interact 2013 – Polemic tweet Question 3: What is the impact of presenting tweets and tags on a web video player? TODO: Figuresp < 0.1 p < 0.1 p < 0.1 0.1 < p < 0.5 0.1 < p < 0.5P > 0.1
  46. 46. CONCLUSION 05/09/2013 IFIP Interact 2013 – Polemic tweet
  47. 47. The Evaluation was challenging 05/09/2013 IFIP Interact 2013 – Polemic tweet   Deployed in the wild   Long period of time   Design change over time   Real life users   Real life setting Because
  48. 48. Result! 1.The system engaged the audience to provide data for crowdsourced video tagging. 2. The visual backchannel incited people to tag their tweets and thus provided an annotation structure. 3. The media player augmented with the annotated tweet visualization was considered a significant improvement over traditional video players.
  49. 49. Next steps 05/09/2013 IFIP Interact 2013 – Polemic tweet As classroom presentations. Synchronous web seminar. Public debates, and social TV.
  50. 50. Possible optimizations: 05/09/2013 IFIP Interact 2013 – Polemic tweet Shared attention. Tag properties. Scalability, user interface design and system.
  51. 51. Subsequent work: 05/09/2013 IFIP Interact 2013 – Polemic tweet http://goo.gl/c2iqg0 Bubble-TV: Live Visual Feedback for Social TV Broadcast Visual Sedimentation: Data stream visualization technique http://goo.gl/KvllnO User supplied sentiment in tweets http://goo.gl/XXVzJb
  52. 52. MORE INFOS: + open source + details THANKS: To people who used the system and people who fund this work: Thank you ! Contact : Samuel.Huron@inria.fr http://www.cybunk.com Team : http://www.aviz.fr http://iri.centrepompidou.fr goo.gl/wKG9Sg
  53. 53. THE END 05/09/2013 IFIP Interact 2013 – Polemic tweet
  54. 54. Survey Result   96% (25) of respondents attended at least one conference had already used twitter at conferences ( 7% only for reading)   86% (24) reported familiarity with the PT syntax   21% (6) with the LiveTweet interface   35% (14) with the web video player visualization timeline. 05/09/2013 WIID: Who Is Interaction Design?
  55. 55. Survey Result In the survey 14 users reported to know what the Tweet Timeline Visualization is. 05/09/2013 WIID: Who Is Interaction Design? • 64% (9) of these found that provides useful information • 79% (11) agree or strongly agree that the visualization helps them to browse the video • 86% (12) would recommend PT to a friend. # to view a part of video with : – many tweets: 57% (8) – positive opinions (++): 29% (4) – negative opinions (–): 43% (6) – questions (??): 21% (3) – references (==): 21% (3)
  56. 56. Cost and benefit of tagging technics 06/09/13 IFIP Interact 2013 – Polemic tweet - Time consuming - Cost is proportional - Hight recall (100%) - Mostly targeted at English - Not “off the shelves” - Precision problem - Cost is proportional - Language agnostic - Accurate - Fast Cost Benefit Mechanical turk - Language agnostic - Accurate - Fast - Cheap, cost is distributed - Engage user - Low recall Natural Language Processing

×