Social Data and Multimedia Analytics for News
and Events Applications
Dr. Yiannis Kompatsiaris, ikom@iti.gr
Multimedia, Kn...
MSDM 2014, Athens Social Data and Multimedia Analytics#2
Overview
• Introduction
– Motivation – Challenges
• SocialSensor ...
MSDM 2014, Athens Social Data and Multimedia Analytics#3
Introduction
Motivation
Example Applications
Conceptual Architect...
MSDM 2014, Athens Social Data and Multimedia Analytics
www.puzzlemarketer.com/digital-social-brands-in-60-seconds/ (Apr, 2...
MSDM 2014, Athens Social Data and Multimedia Analytics
Social Networks as Real-Life Sensors
• Social Networks is a data so...
MSDM 2014, Athens Social Data and Multimedia Analytics#6
Pope Francis
Pope Benedict
2007: iPhone release
2008: Android rel...
MSDM 2014, Athens Social Data and Multimedia Analytics
Social Networks as Graphs
10
social web as a graph
nodes=twi er use...
MSDM 2014, Athens Social Data and Multimedia Analytics#8
Social Networks as Graphs
“Social networks have emergent
properti...
MSDM 2014, Athens Social Data and Multimedia Analytics
Examples - Science
Xin Jin, Andrew Gallagher, Liangliang Cao, Jiebo...
MSDM 2014, Athens Social Data and Multimedia Analytics
Example – News (Boston bombing)
#10
“Following the Boston Marathon ...
MSDM 2014, Athens Social Data and Multimedia Analytics
Events - Festivals
#11
http://www.eventmanagerblog.com/uploads/2012...
MSDM 2014, Athens Social Data and Multimedia Analytics
API Wrapper
Website Wrapper
Scheduler
CRAWLING
Visual Indexing
Near...
MSDM 2014, Athens Social Data and Multimedia Analytics
Challenges – Content (Mining)
• Multi-modality: e.g. image + tags
•...
MSDM 2014, Athens Social Data and Multimedia Analytics
Policy – Licensing – Legal challenges
• Fragmented access to data
–...
MSDM 2014, Athens Social Data and Multimedia Analytics#15
Social Sensor Project
Use Cases
MSDM 2014, Athens Social Data and Multimedia Analytics
SocialSensor Project Objective
SocialSensor quickly surfaces truste...
MSDM 2014, Athens Social Data and Multimedia Analytics#17
The SocialSensor Vision
SocialSensor quickly surfaces trusted an...
MSDM 2014, Athens Social Data and Multimedia Analytics#18
Conceptual Architecture and Main components
SEMANTIC MIDDLEWARE
...
MSDM 2014, Athens Social Data and Multimedia Analytics
Use Cases
Casual News
application
Casual News Readers
Professional
...
MSDM 2014, Athens Social Data and Multimedia Analytics#20
“It has changed the way we do
news”(MSN)
“Social media is the ke...
MSDM 2014, Athens Social Data and Multimedia Analytics#21
Source: Getty Images
“It’s really hard to find the nuggets of us...
MSDM 2014, Athens Social Data and Multimedia Analytics
Verification was simpler in the past...
Source: Frank Grätz
#22
MSDM 2014, Athens Social Data and Multimedia Analytics#23
Infotainment
• Events with large numbers
of visitors
• Thessalon...
MSDM 2014, Athens Social Data and Multimedia Analytics#24
Research Approaches
Large-Scale Visual Search
Clustering – Commu...
MSDM 2014, Athens Social Data and Multimedia Analytics#25
Scalable visual feature aggregation &
indexing
• Problem: Exampl...
MSDM 2014, Athens Social Data and Multimedia Analytics#26
Large-scale visual search
image collection
from social media/
We...
MSDM 2014, Athens Social Data and Multimedia Analytics#27
Framework
• Implementation and evaluation of the effectiveness
o...
MSDM 2014, Athens Social Data and Multimedia Analytics#28
Scalable indexing of features
• ADC 16x8 requires 16 bytes per i...
MSDM 2014, Athens Social Data and Multimedia Analytics#29
VLAD+SIFT vs. VLAD+SURF
Accuracy vs. dimensionality
• VLAD+SURF ...
MSDM 2014, Athens Social Data and Multimedia Analytics
Large-scale graph-based clustering
• Problem: Discover
structure in...
MSDM 2014, Athens Social Data and Multimedia Analytics
• Structural similarity + Local
expansion
(highly efficient and sca...
MSDM 2014, Athens Social Data and Multimedia Analytics
Computational Verification in Social Media
• Create a computational...
MSDM 2014, Athens Social Data and Multimedia Analytics
Methodology
#33
MSDM 2014, Athens Social Data and Multimedia Analytics
Results
• Tweet Statistics
• Approaches
#34
Tweets with URLs 343939...
MSDM 2014, Athens Social Data and Multimedia Analytics
Results(2)
#35
Classifier Classified correctly(%)
Content
features
...
MSDM 2014, Athens Social Data and Multimedia Analytics#36
Other approaches
• Graph-based multimodal clustering for social ...
MSDM 2014, Athens Social Data and Multimedia Analytics#37
Demos - Applications
MM News Demo
Clusttour
ThesFest
MSDM 2014, Athens Social Data and Multimedia Analytics
Multimedia Demo
MSDM 2014, Athens Social Data and Multimedia Analytics#39
Multimedia Demo Architecture
#39
StreamManager
Twitter Facebook ...
MSDM 2014, Athens Social Data and Multimedia Analytics
tags: sagrada familia,
cathedral, barcelona
taken: 12 May 2009
lat:...
MSDM 2014, Athens Social Data and Multimedia Analytics#41
City profile creation (Clusttour)
Community detection on
image s...
MSDM 2014, Athens Social Data and Multimedia Analytics
MSDM 2014, Athens Social Data and Multimedia Analytics#43
ThessFest
• Thessaloniki
International Film
Festival
• Support
t...
MSDM 2014, Athens Social Data and Multimedia Analytics
Fête de la Musique Berlin app
• FETEberlin in App Store and Google ...
MSDM 2014, Athens Social Data and Multimedia Analytics#45
Topic analysis
• Top-10 topics
• Manual inspection
of clusters:
...
MSDM 2014, Athens Social Data and Multimedia Analytics
Other Application Areas
• Science
– Sociology, machine learning (ma...
MSDM 2014, Athens Social Data and Multimedia Analytics
Conclusions – Further topics
• Social media data useful in many app...
MSDM 2014, Athens Social Data and Multimedia Analytics
Reusable results
• Starting point: http://www.socialsensor.eu/resul...
MSDM 2014, Athens Social Data and Multimedia Analytics
European Centre for Social Media
• Topics
– Social media analytics
...
MSDM 2014, Athens Social Data and Multimedia Analytics
Contributions from
• Dr. Symeon Papadopoulos
• Leading R&D in Socia...
Thank you for your attention!
ikom@iti.gr
http://mklab.iti.gr
Upcoming SlideShare
Loading in …5
×

Social Data and Multimedia Analytics for News and Events Applications

906 views

Published on

The keynote discusses a framework enabling real-time multimedia indexing and search across multiple social media sources. It places particular emphasis on the real-time, social and contextual nature of content and information consumption in order to integrate topic and event detection, mining, search and retrieval, based on aggregation and indexing of shared user-generated multimedia content. User-friendly applications for the News and Events domains have been developed based on these approaches, incorporating novel user-centric media visualisation and browsing methods. The research and development is part of the FP7 EU project SocialSensor.

Content:
Introduction
Motivation – Challenges
SocialSensor Project and Use Cases
Research Approaches
Large-Scale visual search
Clustering
Verification
Demos – Applications
MM News Demo
Clusttour
Thessfest
Conclusions

Published in: Technology, Business
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
906
On SlideShare
0
From Embeds
0
Number of Embeds
9
Actions
Shares
0
Downloads
24
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide
  • Benefits: (i) Intelligent extraction of objects and events from the social Web, (ii) multimodal indexing and organization, (iii) personalized access and presentation of content (incl. media delivery and caching), and (iv) concrete and real integration of the social dimension of the current Web.
  • ----- Besprechungsnotizen (03.04.12 14:41) -----
    In the course of the project we have interviewed a considerable number of journalists and executives from some of the worlds biggest media outlets like CNN, the BBC, The New York Times and others... Here are some of the quotes.
    3x klicken (bis alle 3 Quotes sichtbar). But journalists are not only describing the positive side. There are also huge challenges. And you can see from the following slide what is most challenging...
  • ----- Besprechungsnotizen (03.04.12 16:44) -----
    Or we can turn it the other way round: We have a known source whom the reporter trusts...
  • These are all candidate sources for collecting data for the system.
    Ideally, we should try to have at least one source per medium (micro-blogging, photos, videos) + Facebook. Check-ins could be also valuable especially for the WP8 use case.
  • Partners should indicate whether they can make additional data available to the consortium.
  • Social Data and Multimedia Analytics for News and Events Applications

    1. 1. Social Data and Multimedia Analytics for News and Events Applications Dr. Yiannis Kompatsiaris, ikom@iti.gr Multimedia, Knowledge and Social Media Analytics Lab, Head CERTH-ITI Multimodal Social Data Management (MSDM) Workshop
    2. 2. MSDM 2014, Athens Social Data and Multimedia Analytics#2 Overview • Introduction – Motivation – Challenges • SocialSensor Project and Use Cases • Research Approaches – Large-Scale visual search – Clustering – Verification • Demos – Applications – MM News Demo – Clusttour – Thessfest • Conclusions
    3. 3. MSDM 2014, Athens Social Data and Multimedia Analytics#3 Introduction Motivation Example Applications Conceptual Architecture Challenges
    4. 4. MSDM 2014, Athens Social Data and Multimedia Analytics www.puzzlemarketer.com/digital-social-brands-in-60-seconds/ (Apr, 2012)
    5. 5. MSDM 2014, Athens Social Data and Multimedia Analytics Social Networks as Real-Life Sensors • Social Networks is a data source with an extremely dynamic nature that reflects events and the evolution of community focus (user’s interests) • Huge smartphones and mobile devices penetration provides real-time and location-based user feedback • Transform individually rare but collectively frequent media to meaningful topics, events, points of interest, emotional states and social connections • Present in an efficient way for a variety of applications (news, marketing, entertainment)
    6. 6. MSDM 2014, Athens Social Data and Multimedia Analytics#6 Pope Francis Pope Benedict 2007: iPhone release 2008: Android release 2010: iPad release http://petapixel.com/2013/03/14/a-starry-sea-of-cameras-at-the-unveiling-of-pope-francis/
    7. 7. MSDM 2014, Athens Social Data and Multimedia Analytics Social Networks as Graphs 10 social web as a graph nodes=twi er users edges=retweetson #jan25 hashtag announcement of Mubarak’sresigna on h p://gephi.org/2011/the-egyp an-revolu on-on-twi er/
    8. 8. MSDM 2014, Athens Social Data and Multimedia Analytics#8 Social Networks as Graphs “Social networks have emergent properties. Emergent properties are new attributes of a whole that arise from the interaction and interconnection of the parts” •Emotions, Health, Sexual relationships do not depend just on our connections (e.g. number of them) but on our position - structure in the social graph – Central – Hub – Outlier – Transitivity (connections between friends)
    9. 9. MSDM 2014, Athens Social Data and Multimedia Analytics Examples - Science Xin Jin, Andrew Gallagher, Liangliang Cao, Jiebo Luo, and Jiawei Han. The wisdom of social multimedia: using flickr for prediction and forecast, International conference on Multimedia (MM '10). ACM. 9 “…if you're more than 100 km away from the epicenter [of an earthquake] you can read about the quake on twitter before it hits you…”
    10. 10. MSDM 2014, Athens Social Data and Multimedia Analytics Example – News (Boston bombing) #10 “Following the Boston Marathon bombings, one quarter of Americans reportedly looked to Facebook, Twitter and other social networking sites for information, according to The Pew Research Center. When the Boston Police Department posted its final “CAPTURED!!!” tweet of the manhunt, more than 140,000 people retweeted it.” “Authorities have recognized that one the first places people go in events like this is to social media, to see what the crowd is saying about what to do next” "I have been following my friend's Facebook [account] who is near the scene and she is updating everyone before it even gets to the news”
    11. 11. MSDM 2014, Athens Social Data and Multimedia Analytics Events - Festivals #11 http://www.eventmanagerblog.com/uploads/2012/12/event-technology-infographic.jpg
    12. 12. MSDM 2014, Athens Social Data and Multimedia Analytics API Wrapper Website Wrapper Scheduler CRAWLING Visual Indexing Near-duplicates Text Indexing INDEXING Media Fetcher SNA Sentiment - Influence Trends - Topics MINING Model Building Concepts Relevance Diversity Popularity RANKING Veracity Crawling Specs Sources Interaction Responsiveness Aggregation VISUALIZATION Aesthetics Conceptual Architecture
    13. 13. MSDM 2014, Athens Social Data and Multimedia Analytics Challenges – Content (Mining) • Multi-modality: e.g. image + tags • Rich social context: spatio-temporal, social connections, relations and social graph • Inconsistent quality: noise, spam, ambiguity, fake, propaganda • Huge volume: Massively produced and disseminated • Multi-source: may be generated by different applications and user communities • Also connected to other sources (e.g. LOD, web) • Dynamic: Fast updates, real-time
    14. 14. MSDM 2014, Athens Social Data and Multimedia Analytics Policy – Licensing – Legal challenges • Fragmented access to data – Separate wrappers/APIs for each source (Twitter, Facebook, etc.) – Different data collection/crawling policies • Limitations imposed by API providers (“Walled Gardens”) • Full access to data impossible or extremely expensive (e.g. see data licensing plans for GNIP and DataSift • Non-transparent data access practices (e.g. access is provided to an organization/person if they have a contact in Twitter) • Constant change of model and ToS of social APIs – No backwards compatibility, additional development costs • Ephemeral nature of content • Social search results often lead to removed content  inconsistent and unreliable referencing • User Privacy & Purpose of use • Fuzzy regulatory framework regarding mining user-contributed data
    15. 15. MSDM 2014, Athens Social Data and Multimedia Analytics#15 Social Sensor Project Use Cases
    16. 16. MSDM 2014, Athens Social Data and Multimedia Analytics SocialSensor Project Objective SocialSensor quickly surfaces trusted and relevant material from social media – with context. DySCODySCO behaviou r location timecontent usage social context Massive social media and unstructured web Social media mining Aggregation & indexing News - Infotainment Personalised access Ad-hoc P2P networks
    17. 17. MSDM 2014, Athens Social Data and Multimedia Analytics#17 The SocialSensor Vision SocialSensor quickly surfaces trusted and relevant material from social media – with context. •“quickly”: in real time •“surfaces”: automatically discovers, clusters and searches •“trusted”: automatic support in verification process •“relevant”: to the users, personalized •“material”: any material (text, image, audio, video = multimedia), aggregated with other sources (e.g. web) •“social media”: across all relevant social media platforms •“with context”: location, time, sentiment, influence
    18. 18. MSDM 2014, Athens Social Data and Multimedia Analytics#18 Conceptual Architecture and Main components SEMANTIC MIDDLEWARE Public Data In-project Data SEARCH & RECOMMENDATION USER MODELLING & PRESENTATION INDEXINGMINING STORAGE DATA COLLECTION / CRAWLING • Real time dynamic topic and event clustering • Trend, popularity and sentiment analysis • Calculate trust/influence scores around people • Personalized search, access & presentation based on social network interactions • Semantic enrichment and discovery of services
    19. 19. MSDM 2014, Athens Social Data and Multimedia Analytics Use Cases Casual News application Casual News Readers Professional News application Journalists, Editors, etc. NEWS EventLiveDashboard Festival organizers INFOTAINMENT Social Media Walls Festival attendants
    20. 20. MSDM 2014, Athens Social Data and Multimedia Analytics#20 “It has changed the way we do news”(MSN) “Social media is the key place for emerging stories – internationally, nationally, locally” (BBC) “Social media is transforming the way we do journalism” (New York Times) Source: picture alliance / dpa
    21. 21. MSDM 2014, Athens Social Data and Multimedia Analytics#21 Source: Getty Images “It’s really hard to find the nuggets of useful stuff in an ocean of content” (BBC) “Things that aren’t relevant crowd out the content you are looking for” (MSN) “The filters aren’t configurable enough” (CNN)
    22. 22. MSDM 2014, Athens Social Data and Multimedia Analytics Verification was simpler in the past... Source: Frank Grätz #22
    23. 23. MSDM 2014, Athens Social Data and Multimedia Analytics#23 Infotainment • Events with large numbers of visitors • Thessaloniki International Film Festival – 80,000 viewers / 100,000 visitors in 10 days – 150 films, 350 screenings • Discovery and presentation of relevant aggregated social media – Trending Topics – Sentiment – Tweet – film matching – Visualization (Social Walls)
    24. 24. MSDM 2014, Athens Social Data and Multimedia Analytics#24 Research Approaches Large-Scale Visual Search Clustering – Community Detection Social Media Verification
    25. 25. MSDM 2014, Athens Social Data and Multimedia Analytics#25 Scalable visual feature aggregation & indexing • Problem: Example-based image search – Find images that represent same or similar object or scene with a given query image – Viewed from different viewpoints, occlusions, clutter • Challenge: Large-scale – Searching databases with tens of millions of images – Objectives to be full-filed: • Sufficient discriminative power • Fast response times • Efficient memory usage
    26. 26. MSDM 2014, Athens Social Data and Multimedia Analytics#26 Large-scale visual search image collection from social media/ Web image local feature extraction feature aggregation feature indexingkNN visual similarity search concept-based image annotation image clustering image (geo)tagging concept-based search/filtering duplicate detection
    27. 27. MSDM 2014, Athens Social Data and Multimedia Analytics#27 Framework • Implementation and evaluation of the effectiveness of VLAD in combination with SURF • Scalable image indexing E. Spyromitros-Xioufis, et al. An Empirical Study on the Combination of SURF Features with VLAD Vectors for Image Search. In WIAMIS 2012, Dublin, Ireland, May 2012. image local descriptor extraction descriptor aggregation dimensionality reductionset of local descriptors fixed size vector encoding & indexing low dimensional vector SIFT / SURF BOW / VLAD PCA PQ + ADC/IVFADC
    28. 28. MSDM 2014, Athens Social Data and Multimedia Analytics#28 Scalable indexing of features • ADC 16x8 requires 16 bytes per image – ~67M images per GB • IVFADC requires 4 additional bytes per image – ~53.6M images per GB • In current implementation we achieve only half of above numbers due to using short int[] instead of byte[], but possible to improve. • Ideally, 1 billion images could be indexed on a server with 20GB of RAM (projection). • Query time (for 1M vectors): – Exhaustive search of VLAD vectors (d’=128): 0.50 sec – Product Quantization with ADC 16x8: 0.10 sec (x5 faster) – Product Quantization with IVFADC 16x8: 0.02 sec (x25 faster)
    29. 29. MSDM 2014, Athens Social Data and Multimedia Analytics#29 VLAD+SIFT vs. VLAD+SURF Accuracy vs. dimensionality • VLAD+SURF improves VLAD+SIFT and FV+SIFT across all dimensions in both Holidays and Oxford datasets Results in rows starting with * are taken from Jégou et al., 2011, hence the missing values for some entries. SIFT corresponds to PCA reduced SIFT which yielded better results than standard SIFT in Jegou et al., 2011
    30. 30. MSDM 2014, Athens Social Data and Multimedia Analytics Large-scale graph-based clustering • Problem: Discover structure in large-scale datasets by exploiting their relations • Challenges - Approach: – Large-scale – Fast response times – Efficient memory usage – Noise Resilient – Number of clusters not known • Structural similarity + local expansion community detection techniques
    31. 31. MSDM 2014, Athens Social Data and Multimedia Analytics • Structural similarity + Local expansion (highly efficient and scalable approach) • Not necessary to know the number of clusters • Noise resilient (not all nodes need to be part of a community) • Generic approach adaptable to many applications (depending on node – edge representation) + S. Papadopoulos, Y. Kompatsiaris, A. Vakali. “A Graph-based Clustering Scheme for Identifying Related Tags in Folksonomies”. In Proceedings of DaWaK'10, Springer-Verlag, 65-76 Large-scale graph-based clustering
    32. 32. MSDM 2014, Athens Social Data and Multimedia Analytics Computational Verification in Social Media • Create a computational verification framework to classify tweets with unreliable media content. • Events used for experimentation #32 Fake images posted during Hurricane Sandy natural disaster Fake images posted during Boston Marathon bombings
    33. 33. MSDM 2014, Athens Social Data and Multimedia Analytics Methodology #33
    34. 34. MSDM 2014, Athens Social Data and Multimedia Analytics Results • Tweet Statistics • Approaches #34 Tweets with URLs 343939 Tweets with fake images 10758 Tweets with real images 3540 Hurricane Sandy Boston Marathon Tweets with URLs 112449 Tweets with fake images 281 Tweets with real images 460 Classifier Classified correctly(%) Content features User features Total features J48 tree 81.41 67.72 80.68 KStar 81.28 71.16 81.38 Random Forest 80.59 70.15 80.94 Detection accuracy using cross – validation approach Classifier Classified correctly(%) Content features User features Total features J48 tree 76.45 70.81 81.25 KStar 81.28 74.12 75.78 Random Forest 78.59 76.15 79.10 Hurricane Sandy Boston Marathon
    35. 35. MSDM 2014, Athens Social Data and Multimedia Analytics Results(2) #35 Classifier Classified correctly(%) Content features User features Total features J48 tree 73.79 51.06 65.06 KStar 75.30 62.29 53.31 Random Forest 74.02 63.10 65.96 Detection accuracy using different training and testing set in Hurricane Sandy Classifier Classified correctly(%) Content features User features Total features J48 tree 55.05 50.12 54.10 KStar 50.01 50.10 50.97 Random Forest 58.75 51.03 58.78 Detection accuracy using Hurricane Sandy for training and Boston Marathon for testing
    36. 36. MSDM 2014, Athens Social Data and Multimedia Analytics#36 Other approaches • Graph-based multimodal clustering for social event detection in large collections of images – automatic organization of a multimedia collection into groups of items, each (group) of which corresponds to a distinct event. • Unsupervised concept learning detection using social media as training data • Text analysis for entities matching and sentiment analysis • Placing images based on content-features • Retrieving diverse images for same entity
    37. 37. MSDM 2014, Athens Social Data and Multimedia Analytics#37 Demos - Applications MM News Demo Clusttour ThesFest
    38. 38. MSDM 2014, Athens Social Data and Multimedia Analytics Multimedia Demo
    39. 39. MSDM 2014, Athens Social Data and Multimedia Analytics#39 Multimedia Demo Architecture #39 StreamManager Twitter Facebook Flickr YouTube RSS Instagram 160.xx.xx.207 MongoDBWrapper 160.xx.xx.207 TextIndexer (Solr) 160.xx.xx.207 160.xx.xx.207 MediaFetcher, FeatureExtractor (HDFS) 160.xx.xx.58 160.xx.xx.107 Social Focused Crawler (HDFS) 160.xx.xx.187 Nutch Nutch VLAD FeatureIndexer (HDFS) 160.xx.xx.207 IVFADC Data Mining 160.xx.xx.191 Visual Clust. Geo Clust. Statistics Web server 160.xx.xx.116 API (3)API (4) API (1) API (2)
    40. 40. MSDM 2014, Athens Social Data and Multimedia Analytics tags: sagrada familia, cathedral, barcelona taken: 12 May 2009 lat: 41.4036, lon: 2.1743 PHOTOS & METADATA SPATIAL CLUSTERING + TEMPORAL ANALYSIS COMMUNITY DETECTION CLASSIFICATION TO LANDMARKS/EVENTS VISUAL TAG HYBRID [2 years, 50 users / 120 photos] #users / #photos duration [1 day, 2 users / 10 photos] S. Papadopoulos, C. Zigkolis, Y. Kompatsiaris, A. Vakali. “Cluster-based Landmark and Event Detection on Tagged Photo Collections”. In IEEE Multimedia Magazine 18(1), pp. 52-63, 2011 City profile creation (Clusttour)
    41. 41. MSDM 2014, Athens Social Data and Multimedia Analytics#41 City profile creation (Clusttour) Community detection on image similarity graphs Nodes: photos Edges: visual and tag similarity
    42. 42. MSDM 2014, Athens Social Data and Multimedia Analytics
    43. 43. MSDM 2014, Athens Social Data and Multimedia Analytics#43 ThessFest • Thessaloniki International Film Festival • Support twitter/comment usage within the app • Ratings and comments per film • Feedback aggregation • Votes • Tweets • Real-time feedback to the organisation and visitors ThessFest
    44. 44. MSDM 2014, Athens Social Data and Multimedia Analytics Fête de la Musique Berlin app • FETEberlin in App Store and Google Play • More than 100K visitors • About 5K musicians • More than 5K app downloads, 25K sessions App features •Browse and filter detailed program •Interactive maps and routing •Social Sharing •Artists’ and Stages Details •Social Monitoring Main benefits for attendants •Visitors can browse through maps and don’t get lost as stages are numerous •Event schedule is available always and per stage – Very useful when the server was down and there was no access to the online schedule #44
    45. 45. MSDM 2014, Athens Social Data and Multimedia Analytics#45 Topic analysis • Top-10 topics • Manual inspection of clusters: – 53.8% of topic titles considered informative – 98.5% of clusters were found to be “clean” • Topics in time
    46. 46. MSDM 2014, Athens Social Data and Multimedia Analytics Other Application Areas • Science – Sociology, machine learning (machine as a teacher), computer vision (annotation) • Tourism – Leisure – Culture – Off-the-beaten path POI extraction • Marketing – Brand monitoring, personalised ads • Prediction – Politics: election results • News – Topics, trends event detection • Others – Environment, emergency response, energy saving, etc
    47. 47. MSDM 2014, Athens Social Data and Multimedia Analytics Conclusions – Further topics • Social media data useful in many applications • Not all data always available (e.g. User queries, fb) – Infrastructure – Policy - Privacy issues • Real-time and scalable approaches – Efficiency of semantics and analysis vs. performance vs. infrastructure • Fusion of various modalities – Content, social, temporal, location • Verification & Linking other sources (web, Linked Open Data) • Visualization - Interfaces • Applications and commercialization • User engagement
    48. 48. MSDM 2014, Athens Social Data and Multimedia Analytics Reusable results • Starting point: http://www.socialsensor.eu/results – Deliverables – Publications – Datasets – Software – e-letter: http://stcsn.ieee.net/e-letter/vol-1-no-3 • Open-source projects (Apache License v2): https://github.com/socialsensor – Data collection (stream-manager, storm-focused-crawler) – Indexing (framework-client, multimedia-indexing) – Mining (topic-detection, multimedia-analysis, community-evolution- analysis, social-event-detection)
    49. 49. MSDM 2014, Athens Social Data and Multimedia Analytics European Centre for Social Media • Topics – Social media analytics – Verification – Visualisation – Applications in different domains • Activities – Listings of project, results, institutions, events – Community building – Support/organise events – Common social media presence (e.g. LinkedIn) – Funding from subscriptions, training, commercialisation – Supporting projects: SocialSensor, Reveal, MULTISENSOR, PHEME, DecarboNet, MWCC, uComp, – Website: http://www.socialmediacentre.eu/ – Research-academic: STCSN http://stcsn.ieee.net/
    50. 50. MSDM 2014, Athens Social Data and Multimedia Analytics Contributions from • Dr. Symeon Papadopoulos • Leading R&D in Social Media Mining • Large-Scale visual search • Community detection – Clusttour • Dr. Sotirios Diplaris • SocialSensor Technical Project Manager • Lefteris Spyromitros (PhD Student, AUTH) • Large-Scale visual search • Christina Boididou • Social Media Verification • Lazaros Apostolidis • Visualization - User Interface MM News Dem0 • Manos Schinas • Topic Analysis • Back-end Thessfest – Clusttour • MM News Demo • Juxhin Bakalli • iOS Applications development (ThessFest - Clusttour) • Antonis Latas • Android Application Development (Thessfest)
    51. 51. Thank you for your attention! ikom@iti.gr http://mklab.iti.gr

    ×