The keynote discusses a framework enabling real-time multimedia indexing and search across multiple social media sources. It places particular emphasis on the real-time, social and contextual nature of content and information consumption in order to integrate topic and event detection, mining, search and retrieval, based on aggregation and indexing of shared user-generated multimedia content. User-friendly applications for the News and Events domains have been developed based on these approaches, incorporating novel user-centric media visualisation and browsing methods. The research and development is part of the FP7 EU project SocialSensor.
Content:
Introduction
Motivation – Challenges
SocialSensor Project and Use Cases
Research Approaches
Large-Scale visual search
Clustering
Verification
Demos – Applications
MM News Demo
Clusttour
Thessfest
Conclusions
Hashtag Conversations,Eventgraphs, and User Ego Neighborhoods: Extracting So...Shalin Hai-Jew
This introduces methods for extracting and analyzing social network data from Twitter for hashtag conversations (and emergent events), event graphs, search networks, and user ego neighborhoods (using NodeXL). There will be direct demonstrations and discussions of how to analyze social network graphs. This information may be extended with human- and / or machine-based sentiment analysis.
Understanding Public Sentiment: Conducting a Related-Tags Content Network Ext...Shalin Hai-Jew
This presentation focuses on how to understand public sentiment through a related-tags content network analysis of public Flickr photos and videos. NodeXL is used to conduct data extractions and visualizations of user-tagged Flickr contents and the resulting “noisy” folksonomies. What mental connections may be made about particular issues based on analysis of text-annotated graphs?
Invited talk presented by Hemant Purohit (http://knoesis.org/researchers/hemant) at the NCSU workshop on IT for sustainable tourism development. The talk presents application of technology developed for crisis coordination into more general marketplace coordination via social media for helping suppliers (micro-entrepreneurs) and demanders (tourists).
Coding Social Imagery: Learning from a #selfie #humor Image Set from InstagramShalin Hai-Jew
Social media messaging has long been harnessed to inform faculty about their respective learners. The textual channel is often used because of the ease of interpretation and analysis. Social imagery—tagged images, #selfies, grouped imagery, and others—has been less used, in part because images are more complex and multi-meaninged to analyze. Also, there are not many generalist models that inform how to code or even understand social imagery in an emergent way. (There are large-scale computational means to interpret online images, such as the AlchemyAPI of IBM Watson, for various types of feature extractions. There are ways to code imagery based on specific research questions in particular fields-of-practice.)
The presenter recently analyzed a 941-image #selfie + #humor image set from Instagram, with three main research questions:
What does identity-based humor look like in terms of a #selfie #humor- tagged image set from the Instagram photo-sharing mobile app?
Do more modern forms of mediated social humor link to more traditional forms theoretically? Is it possible to apply the Humor Styles Model to the images from the #selfie #humor Instagram image set to better understand #selfie #humor?
What are some constructive and systematized ways to analyze social image sets manually (with some computational support)?
This digital poster session will highlight some of the initial research findings (forthcoming in a near-future publication) and share insights about effectively coding social imagery in a bottom-up and emergent way.
Social media mining presentation by Tom Masterman of the Institute on the Environment, given at the University of Minnesota's Communicators Forum Conference, 05.13.10.
Hashtag Conversations,Eventgraphs, and User Ego Neighborhoods: Extracting So...Shalin Hai-Jew
This introduces methods for extracting and analyzing social network data from Twitter for hashtag conversations (and emergent events), event graphs, search networks, and user ego neighborhoods (using NodeXL). There will be direct demonstrations and discussions of how to analyze social network graphs. This information may be extended with human- and / or machine-based sentiment analysis.
Understanding Public Sentiment: Conducting a Related-Tags Content Network Ext...Shalin Hai-Jew
This presentation focuses on how to understand public sentiment through a related-tags content network analysis of public Flickr photos and videos. NodeXL is used to conduct data extractions and visualizations of user-tagged Flickr contents and the resulting “noisy” folksonomies. What mental connections may be made about particular issues based on analysis of text-annotated graphs?
Invited talk presented by Hemant Purohit (http://knoesis.org/researchers/hemant) at the NCSU workshop on IT for sustainable tourism development. The talk presents application of technology developed for crisis coordination into more general marketplace coordination via social media for helping suppliers (micro-entrepreneurs) and demanders (tourists).
Coding Social Imagery: Learning from a #selfie #humor Image Set from InstagramShalin Hai-Jew
Social media messaging has long been harnessed to inform faculty about their respective learners. The textual channel is often used because of the ease of interpretation and analysis. Social imagery—tagged images, #selfies, grouped imagery, and others—has been less used, in part because images are more complex and multi-meaninged to analyze. Also, there are not many generalist models that inform how to code or even understand social imagery in an emergent way. (There are large-scale computational means to interpret online images, such as the AlchemyAPI of IBM Watson, for various types of feature extractions. There are ways to code imagery based on specific research questions in particular fields-of-practice.)
The presenter recently analyzed a 941-image #selfie + #humor image set from Instagram, with three main research questions:
What does identity-based humor look like in terms of a #selfie #humor- tagged image set from the Instagram photo-sharing mobile app?
Do more modern forms of mediated social humor link to more traditional forms theoretically? Is it possible to apply the Humor Styles Model to the images from the #selfie #humor Instagram image set to better understand #selfie #humor?
What are some constructive and systematized ways to analyze social image sets manually (with some computational support)?
This digital poster session will highlight some of the initial research findings (forthcoming in a near-future publication) and share insights about effectively coding social imagery in a bottom-up and emergent way.
Social media mining presentation by Tom Masterman of the Institute on the Environment, given at the University of Minnesota's Communicators Forum Conference, 05.13.10.
Eavesdropping on the Twitter Microblogging SiteShalin Hai-Jew
Research analysts go to Twitter to capture the general trends of public conversations, identify and profile influential accounts, and extract subgroups within larger collectives and larger discourses; they also go to eavesdrop on individual self-talk and individual-to-individual conversations. So what is technically in your tweets, asked Dave Rosenberg famously in a CNET article (2010). The answer: a whole lot more than 140 characters. How are the most influential social media accounts identified through #hashtag graphs? How are themes extracted? How are sentiments understood? How can users be profiled through their Tweetstreams? How can locations be mapped in terms of the Twitter conversations occurring in particular physical areas? How can live and trending issues be identified and categorized in terms of sentiment (positive, negative, and neutral)? This presentation will summarize some of the free and open-source tools as well as commercial and proprietary ones that enable increased knowability.
"Geoparsing and Real-time Social Media Analytics - technical and social challenges"
UK ESRC seminar series - Microenterprise, technology and big data.
Southampton, UK. Stuart E. Middleton (ITINNO) presented the REVEAL project to the UK social science research community.
Social Media & Web Mining for Public Services of Smart Cities - SSA TalkHemant Purohit
This talk at Data Science Seminar of SSA presents challenges and methods to model behavior on social media & Web for application opportunities for public services. The talk also demonstrates an in-depth case study of mining intentional behavior from the noisy natural language text of social media messages during disasters and how it could assist emergency services of future smart cities.
Given the growth of social media and rapid evolution of Web of Data, we have unprecedented opportunities to improve crisis response by extracting social signals, creating spatio-temporal mappings, performing analytics on social and Web of Data, and supporting a variety of applications. Such applications can help provide situational awareness during an emergency, improve preparedness, and assist during the rebuilding/recovery phase of a disaster. Data mining can provide valuable insights to support emergency responders and other stakeholders during crisis. However, there are a number of challenges and existing computing technology may not work in all cases. Therefore, our objective here is to present the characterization of such data mining tasks, and challenges that need further research attention for leveraging social media and Web of Data to assist crisis response coordination.
30 Tools and Tips to Speed Up Your Digital Workflow Mike Kujawski
Have you ever found yourself wasting a considerable amount of time performing some annoying, repetitive process within a common application, social media website, or your web browser? Wish there was a "magic" shortcut or simply a better way of getting it done? There most likely is.
While having a solid strategy should always be the first priority before engaging in the digital/social media space, it's also smart to arm yourself with a set of tools that will help you with the tactical implementation of your plan. These presentation slides provide 30 tools and tips hand-picked from Mike Kujawski's personal experience, day-to-day observations, and interactions with his consulting and training clients.
These tools are meant to help you be more efficient and effective as a communicator in today's digital world, where agility and "life-hacking" skills are becoming increasingly valued.
What Actor-Network Theory (ANT) and digital methods can do for data journalis...Liliana Bounegru
Slides from a talk I gave at the University of Ghent on 21 October 2014 about how Actor-Network Theory (ANT) and digital methods can be used to study and inform data journalism.
Invited talk at Session on Semantic Knowledge for Commodity Computing, at Microsoft Research Faculty Summit 2011, July 19-20, 2011, Redmond, WA. http://research.microsoft.com/en-us/events/fs2011/default.aspx
Associated video at: https://youtu.be/HKqpuLiMXRs
Journalists today are faced with an overwhelming abundance of data – from large collections of leaked documents, to public databases about lobbying or government spending, to ‘big data’ from social networks such as Twitter and Facebook. To stay relevant to society journalists are learning to process this data and separate signal from noise in order to provide valuable insights to their readers. This talk will address questions like: What is the potential of data journalism? Why is it relevant to society? And how can you get started?
Socialsensor project overview and topic discovery in tweeter streams Yiannis Kompatsiaris
A description of SocialSensor project motivation, objectives and use cases in news and infotainment. First results in topic discovery in tweeter streams.
Eavesdropping on the Twitter Microblogging SiteShalin Hai-Jew
Research analysts go to Twitter to capture the general trends of public conversations, identify and profile influential accounts, and extract subgroups within larger collectives and larger discourses; they also go to eavesdrop on individual self-talk and individual-to-individual conversations. So what is technically in your tweets, asked Dave Rosenberg famously in a CNET article (2010). The answer: a whole lot more than 140 characters. How are the most influential social media accounts identified through #hashtag graphs? How are themes extracted? How are sentiments understood? How can users be profiled through their Tweetstreams? How can locations be mapped in terms of the Twitter conversations occurring in particular physical areas? How can live and trending issues be identified and categorized in terms of sentiment (positive, negative, and neutral)? This presentation will summarize some of the free and open-source tools as well as commercial and proprietary ones that enable increased knowability.
"Geoparsing and Real-time Social Media Analytics - technical and social challenges"
UK ESRC seminar series - Microenterprise, technology and big data.
Southampton, UK. Stuart E. Middleton (ITINNO) presented the REVEAL project to the UK social science research community.
Social Media & Web Mining for Public Services of Smart Cities - SSA TalkHemant Purohit
This talk at Data Science Seminar of SSA presents challenges and methods to model behavior on social media & Web for application opportunities for public services. The talk also demonstrates an in-depth case study of mining intentional behavior from the noisy natural language text of social media messages during disasters and how it could assist emergency services of future smart cities.
Given the growth of social media and rapid evolution of Web of Data, we have unprecedented opportunities to improve crisis response by extracting social signals, creating spatio-temporal mappings, performing analytics on social and Web of Data, and supporting a variety of applications. Such applications can help provide situational awareness during an emergency, improve preparedness, and assist during the rebuilding/recovery phase of a disaster. Data mining can provide valuable insights to support emergency responders and other stakeholders during crisis. However, there are a number of challenges and existing computing technology may not work in all cases. Therefore, our objective here is to present the characterization of such data mining tasks, and challenges that need further research attention for leveraging social media and Web of Data to assist crisis response coordination.
30 Tools and Tips to Speed Up Your Digital Workflow Mike Kujawski
Have you ever found yourself wasting a considerable amount of time performing some annoying, repetitive process within a common application, social media website, or your web browser? Wish there was a "magic" shortcut or simply a better way of getting it done? There most likely is.
While having a solid strategy should always be the first priority before engaging in the digital/social media space, it's also smart to arm yourself with a set of tools that will help you with the tactical implementation of your plan. These presentation slides provide 30 tools and tips hand-picked from Mike Kujawski's personal experience, day-to-day observations, and interactions with his consulting and training clients.
These tools are meant to help you be more efficient and effective as a communicator in today's digital world, where agility and "life-hacking" skills are becoming increasingly valued.
What Actor-Network Theory (ANT) and digital methods can do for data journalis...Liliana Bounegru
Slides from a talk I gave at the University of Ghent on 21 October 2014 about how Actor-Network Theory (ANT) and digital methods can be used to study and inform data journalism.
Invited talk at Session on Semantic Knowledge for Commodity Computing, at Microsoft Research Faculty Summit 2011, July 19-20, 2011, Redmond, WA. http://research.microsoft.com/en-us/events/fs2011/default.aspx
Associated video at: https://youtu.be/HKqpuLiMXRs
Journalists today are faced with an overwhelming abundance of data – from large collections of leaked documents, to public databases about lobbying or government spending, to ‘big data’ from social networks such as Twitter and Facebook. To stay relevant to society journalists are learning to process this data and separate signal from noise in order to provide valuable insights to their readers. This talk will address questions like: What is the potential of data journalism? Why is it relevant to society? And how can you get started?
Socialsensor project overview and topic discovery in tweeter streams Yiannis Kompatsiaris
A description of SocialSensor project motivation, objectives and use cases in news and infotainment. First results in topic discovery in tweeter streams.
SocialSensor Project: Sensing User Generated Input for Improved Media Discove...Yiannis Kompatsiaris
SocialSensor: Sensing User Generated Input for Improved Media Discovery and Experience
Social Multimedia Crawling & Mining
EventSense: Capturing the Pulse of Large-scale Events by Mining Social Media Streams
Citizen Sensor Data Mining, Social Media Analytics and ApplicationsAmit Sheth
Opening talk at Singapore Symposium on Sentiment Analysis (S3A), February 6, 2015, Singapore. http://s3a.sentic.net/#s3a2015
Abstract
With the rapid rise in the popularity of social media, and near ubiquitous mobile access, the sharing of observations and opinions has become common-place. This has given us an unprecedented access to the pulse of a populace and the ability to perform analytics on social data to support a variety of socially intelligent applications -- be it for brand tracking and management, crisis coordination, organizing revolutions or promoting social development in underdeveloped and developing countries.
I will review: 1) understanding and analysis of informal text, esp. microblogs (e.g., issues of cultural entity extraction and role of semantic/background knowledge enhanced techniques), and 2) how we built Twitris, a comprehensive social media analytics (social intelligence) platform.
I will describe the analysis capabilities along three dimensions: spatio-temporal-thematic, people-content-network, and sentiment-emption-intent. I will couple technical insights with identification of computational techniques and real-world examples using live demos of Twitris (http://twitris2.knoesis.org).
Researching Social Media – Big Data and Social Media AnalysisFarida Vis
Researching Social Media – Big Data and Social Media Analysis, presentation for the Social Media for Researchers: A Sheffield Universities Social Media Symposium, 23 September 2014
From Research to Applications: What Can We Extract with Social Media Sensing?Yiannis Kompatsiaris
SIGMAP22 Keynote Presentation:
Social media have transformed the Web into an interactive sharing platform where users upload data and media, comment on, and share this content within their social circles. The large-scale availability of user-generated content in social media platforms has opened up new possibilities for studying and understanding real-world phenomena, trends and events. Social media and websites provide an access to public opinions on certain aspects and therefore play an important role in getting insights on targeted audiences. The objective of this talk is to provide an overview of social media mining, including key aspects such as data collection, multimodal analysis and visualization. Challenges, such as fighting misinformation, will be presented together with applications, results and demonstrations from multiple areas including: news, environment, security, interior and urban design.
Picturing the Social: Talk for Transforming Digital Methods Winter SchoolFarida Vis
This talk highlights the work of the Visual Social Media Lab and the Picturing the Social project. It summarises the key research questions and aims of the project. It highlights the value of interdisciplinarity and working closely with industry in this area. It also focuses on the way in which me might study different types of structures involved in the circulation and the scopic regimes that make social media images more or less visible. It also tries to unpack how we can start to think about APIs as 'method' and looks at the different ways in which we can get access to different kinds of social media image data. Both through public ('free') APIs and ('pay for') firehose data.
Visual Information Analysis for Crisis and Natural Disasters Management and R...Yiannis Kompatsiaris
Invited talk at the Ninth International Conference on Image Processing Theory, Tools and Applications IPTA 2019 (http://www.ipta-conference.com/ipta19/)
Crises and natural disasters are unwelcome, but also unavoidable features of modern society, affecting more communities than ever. Visual information analysis plays an important role in efficient pre-event (e.g. risk modeling), during the event (response) and post-event (recovery) emergency situation management. This talk will describe the potential role of visual information sources including satellite images, surveillance and traffic cameras, social multimedia and aerial video in applications such as floods, fires, and oil spills. Multimodal and fusion techniques will be presented combining satellite and social data and how deep neural networks can be applied in this domain. The talks will include demos and results from the relevant BeAware and EOPEN projects and from our participation in the 2018 Multimedia Satellite Task of the MediaEval Benchmarking Initiative.
This is a brief a brief review of current multi-disciplinary and collaborative projects at Kno.e.sis led by Prof. Amit Sheth. They cover research in big social data, IoT, semantic web, semantic sensor web, health informatics, personalized digital health, social data for social good, smart city, crisis informatics, digital data for material genome initiative, etc. Dec 2015 edition.
The workshop opens with a discussion of how to repurpose digital "methods of the medium" for social and cultural scholarly research, including its limitations, critiques and ethics. Subsequently participants are trained in using digital methods in hands-on sessions. How to use crawlers for dynamic URL sampling and issue network mapping? How to employ scrapers to create a bias or partisanship diagnostic instrument? We also consider how to deploy online platforms for social research. How to transform Wikipedia from an online encyclopaedia to a device for cross-cultural memory studies? How to make use of social media so as to profile the preferences and tastes of politicians’ friends, and also locate most engaged with content? How to make use of Twitter analytics to debanalize tweets, and provide compelling accounts of events on the ground? Finally, the workshop turns to the question of employing web data and metrics as societal indices more generally.
Objectives: 1. Gain an understanding of key trends in ICT innovation which are influencing/disrupting crisis informatics. 2. Be able to trace these trends through discussions later this semester, and understand their influence and potential. 3. Introduce visualization lab
Working with Social Media Data: Ethics & good practice around collecting, usi...Nicola Osborne
Slides from a workshop delivered for the University of Edinburgh Digital Scholarship programme, on 18th October 2017. For further information on the programme see: http://www.digital.cahss.ed.ac.uk/ or #DigScholEd. If you are interested in hosting a similar workshop, or adapting these slides please contact me: nicola.osborne@ed.ac.uk.
Similar to Social Data and Multimedia Analytics for News and Events Applications (20)
Social media mining for sensing and responding to real-world trends and eventsYiannis Kompatsiaris
Social media have transformed the Web into an interactive sharing platform where users upload data and media, comment on, and share this content within their social circles. The large-scale availability of user-generated content in social media platforms has opened up new possibilities for studying and understanding real-world phenomena, trends and events. The objective of this talk is to provide an overview of social media mining, which offers a unique opportunity to to discover, collect, and extract relevant information in order to provide useful insights. It will include key challenges and issues, such as fighting misinformation, data collection, analysis and visualization components, applications, results and demos from multiple areas ranging from news to environmental and security ones.
This is a presentation on Sensor Based Ambient Assisted Living architecture and approaches developed by the Multimedia Knowledge and Social Media Analytics Lab of CERTH-ITI. It includes sensors used for monitoring Activities of Daily Living of elders and persons with mild Dementia at home. Visual and sensor data analytics are combined with formal representations (ontology), fusion, reasoning techniques and visualizations in order to provide an objective view of everyday activities. Example projects and pilots are included. Clinical assessment show improvement in cognitive abilities of participants.
The pervasive use of Online Social Networks (OSN) for networking, communication and search in tandem with the ubiquitous availability of smartphones, which enables real-time multimedia capturing and sharing, have led to massive amounts of user-generated content and activities being amassed online, and made publicly available for analysis and mining. Each content item is associated with an abundance of metadata and related information such as location, tags, comments, favorites and mood indicators, access logs, and so on. At the same time, all this information is implicitly or explicitly interconnected based on various properties such as social links among users, groups, communities, and sharing patterns. These properties transform social media into data sources of an extremely dynamic nature that reflect topics of interests, events, and the evolution of community opinion and focus. Social media processing offers a unique opportunity to structure and extract information and to benefit multiple areas ranging from new media experiences to psychology and marketing. The objective of this talk is to provide an overview of the current research in emerging topics related to applications where social media can act as sensors of real-life phenomena and case studies that reveal valuable insights. After discussing challenges and presenting a generic conceptual architecture, there will be a focus on efficient processing and indexing algorithms that can handle massive amounts of content with application to graph-based event detection and summarization in social media streams.
Social Media Verification Challenges, Approaches and ApplicationsYiannis Kompatsiaris
As grassroots and social media-based journalism becomes more widespread, the need to verify information coming from such channels becomes imperative. The objective of this talk is to explore the challenges involved in social media computational verification to automatically classify unreliable media content as fake or real. After presenting a generic conceptual architecture, there will be a focus on tweets around big events linking to images (fake or real) of which the reliability could be verified by independent online sources. The REVEALr platform will be demonstrated, a scalable and efficient content-based media crawling and indexing framework featuring a novel and resilient near-duplicate detection approach and intelligent content- and context-based aggregation capabilities (e.g. clustering, named entity extraction)
Social Data and Multimedia Analytics for News and Events Applications lecture given at 2015 IEEE SPS Italy Chapter Summer School on Signal Processing (S3P)
The DemaWare Service-Oriented AAL Platform for People with DementiaYiannis Kompatsiaris
This work presents DemaWare, an Ambient Intelligence platform that targets Ambient Assisted Living for people with Dementia. DemaWare seamlessly integrates diverse hardware (wearable and ambient sensors), as well as soft- ware components (semantic interpretation, reasoning), involved in such context. It also enables both online and offline processes, including sensor analysis and storage of context semantics in a Knowledge Base. Consequently, it orchestrates semantic interpretation which incorporated defeasible logics for uncertainty handling. Overall, the underlying functionality aids clinicians and carers to timely assess and diagnose patients in the context of lab trials, homes or nursing homes.
"Μια πόλη από το μέλλον": Πως ο πολίτης μπορεί να γίνει συμμέτοχος μέσω της χ...Yiannis Kompatsiaris
Παρουσιάζονται παραδείγματα και αποτελέσματα του Εργαστηρίου Γνώσης, Πολυμέσων και Ανάλυσης Κοινωνικών Δικτύων του ΕΚΕΤΑ-ΙΠΤΗΛ (http://mklab.iti.gr) που αφορούν εφαρμογές που υποστηρίζουν τη συμμετοχή των πολιτών σε τομείς όπως: Αυτο-οργάνωση εθελοντικών κινημάτων, Συλλογική καταγραφή προβλημάτων, Συμμετοχή στη διαχείριση προβλημάτων και δημοσίου χώρου, Σχεδιασμό σε μελλοντικές δράσεις και έργα στην πόλη. Η παρουσίαση δόθηκε στα πλαίσια του συνεδρίου "Μια πόλη από το μέλλον", που διοργανώθηκε από την Παράλλαξη: http://www.parallaximag.gr/parallax-view/elate-na-deite-mia-poli-apo-mellon
Τεχνικές Αναγνώρισης Προτύπων και Μηχανικής Μάθησης για Εφαρμογές Ανάλυσης Πο...Yiannis Kompatsiaris
Η ομιλία περιλαμβάνει εφαρμογές τεχνικών αναγνώρισης προτύπων και μηχανικής μάθησης σε ανάλυση πολυμέσων και κοινωνικών δικτύων. Πιο συγκεκριμένα, θα παρουσιαστούν τεχνικές και εφαρμογές κατάτμησης εικόνων με χρήση Κ-Μέσων και επεκτάσεων, χρήση Support Vector Ma-chines για μάθηση εννοιών σε εικόνες καθώς και τεχνικές ανάλυσης γράφων από κοινωνικά δίκτυα. Θα παρουσιαστούν σχετικές εφαρμογές που αξιοποιούν τα αποτελέσματα της ανάλυσης, όπως αναζήτηση πολυμέσων και εφαρμογή για τουρισμό και ενημέρωση από κοινωνικά δίκτυα. Θα αναφερθούν τρέχοντα ερευνητικά προβλήματα και περιοχές.
Improve My City: App for Citizens Reporting Issues in Municipalities – RegionsYiannis Kompatsiaris
The application enables citizens to report local problems such as potholes, illegal trash dumping, faulty street lights, broken tiles on sidewalks, and illegal advertising boards. The submitted issues are displayed on the city's map. Users may add photos and comments. Moreover, they can suggest solutions for improving the environment of their neighbourhood.
Social media mining and multimedia analysis research and applicationsYiannis Kompatsiaris
In this talk, research and applications in social media mining and multimedia analysis are going to be presented. Social media sharing websites host billions of images and videos, which have been annotated and shared among friends, or published in groups that cover a specific topic of interest. The fact that users annotate and comment on the content in the form of tags, ratings, preferences and so on, and that these activities are performed on a daily basis, gives such social media data source an extremely dynamic nature that reflects topics of interests, events and the evolution of community opinion and focus.
The talk will present research challenges and activities and will focus on multi-modal graph-based community detection methods for social media mining, concept and event detection. Clusttour, a mobile and web application integrating research results with appropriate interface design will be demonstrated as a relevant use case. The talk will also include approaches for object/region classifiers learned using the self-training paradigm with loosely annotated training samples automatically selected from social media.
Full-RAG: A modern architecture for hyper-personalizationZilliz
Mike Del Balso, CEO & Co-Founder at Tecton, presents "Full RAG," a novel approach to AI recommendation systems, aiming to push beyond the limitations of traditional models through a deep integration of contextual insights and real-time data, leveraging the Retrieval-Augmented Generation architecture. This talk will outline Full RAG's potential to significantly enhance personalization, address engineering challenges such as data management and model training, and introduce data enrichment with reranking as a key solution. Attendees will gain crucial insights into the importance of hyperpersonalization in AI, the capabilities of Full RAG for advanced personalization, and strategies for managing complex data integrations for deploying cutting-edge AI solutions.
GraphSummit Singapore | The Future of Agility: Supercharging Digital Transfor...Neo4j
Leonard Jayamohan, Partner & Generative AI Lead, Deloitte
This keynote will reveal how Deloitte leverages Neo4j’s graph power for groundbreaking digital twin solutions, achieving a staggering 100x performance boost. Discover the essential role knowledge graphs play in successful generative AI implementations. Plus, get an exclusive look at an innovative Neo4j + Generative AI solution Deloitte is developing in-house.
UiPath Test Automation using UiPath Test Suite series, part 6DianaGray10
Welcome to UiPath Test Automation using UiPath Test Suite series part 6. In this session, we will cover Test Automation with generative AI and Open AI.
UiPath Test Automation with generative AI and Open AI webinar offers an in-depth exploration of leveraging cutting-edge technologies for test automation within the UiPath platform. Attendees will delve into the integration of generative AI, a test automation solution, with Open AI advanced natural language processing capabilities.
Throughout the session, participants will discover how this synergy empowers testers to automate repetitive tasks, enhance testing accuracy, and expedite the software testing life cycle. Topics covered include the seamless integration process, practical use cases, and the benefits of harnessing AI-driven automation for UiPath testing initiatives. By attending this webinar, testers, and automation professionals can gain valuable insights into harnessing the power of AI to optimize their test automation workflows within the UiPath ecosystem, ultimately driving efficiency and quality in software development processes.
What will you get from this session?
1. Insights into integrating generative AI.
2. Understanding how this integration enhances test automation within the UiPath platform
3. Practical demonstrations
4. Exploration of real-world use cases illustrating the benefits of AI-driven test automation for UiPath
Topics covered:
What is generative AI
Test Automation with generative AI and Open AI.
UiPath integration with generative AI
Speaker:
Deepak Rai, Automation Practice Lead, Boundaryless Group and UiPath MVP
Communications Mining Series - Zero to Hero - Session 1DianaGray10
This session provides introduction to UiPath Communication Mining, importance and platform overview. You will acquire a good understand of the phases in Communication Mining as we go over the platform with you. Topics covered:
• Communication Mining Overview
• Why is it important?
• How can it help today’s business and the benefits
• Phases in Communication Mining
• Demo on Platform overview
• Q/A
LF Energy Webinar: Electrical Grid Modelling and Simulation Through PowSyBl -...DanBrown980551
Do you want to learn how to model and simulate an electrical network from scratch in under an hour?
Then welcome to this PowSyBl workshop, hosted by Rte, the French Transmission System Operator (TSO)!
During the webinar, you will discover the PowSyBl ecosystem as well as handle and study an electrical network through an interactive Python notebook.
PowSyBl is an open source project hosted by LF Energy, which offers a comprehensive set of features for electrical grid modelling and simulation. Among other advanced features, PowSyBl provides:
- A fully editable and extendable library for grid component modelling;
- Visualization tools to display your network;
- Grid simulation tools, such as power flows, security analyses (with or without remedial actions) and sensitivity analyses;
The framework is mostly written in Java, with a Python binding so that Python developers can access PowSyBl functionalities as well.
What you will learn during the webinar:
- For beginners: discover PowSyBl's functionalities through a quick general presentation and the notebook, without needing any expert coding skills;
- For advanced developers: master the skills to efficiently apply PowSyBl functionalities to your real-world scenarios.
Unlock the Future of Search with MongoDB Atlas_ Vector Search Unleashed.pdfMalak Abu Hammad
Discover how MongoDB Atlas and vector search technology can revolutionize your application's search capabilities. This comprehensive presentation covers:
* What is Vector Search?
* Importance and benefits of vector search
* Practical use cases across various industries
* Step-by-step implementation guide
* Live demos with code snippets
* Enhancing LLM capabilities with vector search
* Best practices and optimization strategies
Perfect for developers, AI enthusiasts, and tech leaders. Learn how to leverage MongoDB Atlas to deliver highly relevant, context-aware search results, transforming your data retrieval process. Stay ahead in tech innovation and maximize the potential of your applications.
#MongoDB #VectorSearch #AI #SemanticSearch #TechInnovation #DataScience #LLM #MachineLearning #SearchTechnology
zkStudyClub - Reef: Fast Succinct Non-Interactive Zero-Knowledge Regex ProofsAlex Pruden
This paper presents Reef, a system for generating publicly verifiable succinct non-interactive zero-knowledge proofs that a committed document matches or does not match a regular expression. We describe applications such as proving the strength of passwords, the provenance of email despite redactions, the validity of oblivious DNS queries, and the existence of mutations in DNA. Reef supports the Perl Compatible Regular Expression syntax, including wildcards, alternation, ranges, capture groups, Kleene star, negations, and lookarounds. Reef introduces a new type of automata, Skipping Alternating Finite Automata (SAFA), that skips irrelevant parts of a document when producing proofs without undermining soundness, and instantiates SAFA with a lookup argument. Our experimental evaluation confirms that Reef can generate proofs for documents with 32M characters; the proofs are small and cheap to verify (under a second).
Paper: https://eprint.iacr.org/2023/1886
Observability Concepts EVERY Developer Should Know -- DeveloperWeek Europe.pdfPaige Cruz
Monitoring and observability aren’t traditionally found in software curriculums and many of us cobble this knowledge together from whatever vendor or ecosystem we were first introduced to and whatever is a part of your current company’s observability stack.
While the dev and ops silo continues to crumble….many organizations still relegate monitoring & observability as the purview of ops, infra and SRE teams. This is a mistake - achieving a highly observable system requires collaboration up and down the stack.
I, a former op, would like to extend an invitation to all application developers to join the observability party will share these foundational concepts to build on:
GridMate - End to end testing is a critical piece to ensure quality and avoid...ThomasParaiso2
End to end testing is a critical piece to ensure quality and avoid regressions. In this session, we share our journey building an E2E testing pipeline for GridMate components (LWC and Aura) using Cypress, JSForce, FakerJS…
UiPath Test Automation using UiPath Test Suite series, part 5DianaGray10
Welcome to UiPath Test Automation using UiPath Test Suite series part 5. In this session, we will cover CI/CD with devops.
Topics covered:
CI/CD with in UiPath
End-to-end overview of CI/CD pipeline with Azure devops
Speaker:
Lyndsey Byblow, Test Suite Sales Engineer @ UiPath, Inc.
Threats to mobile devices are more prevalent and increasing in scope and complexity. Users of mobile devices desire to take full advantage of the features
available on those devices, but many of the features provide convenience and capability but sacrifice security. This best practices guide outlines steps the users can take to better protect personal devices and information.
Removing Uninteresting Bytes in Software FuzzingAftab Hussain
Imagine a world where software fuzzing, the process of mutating bytes in test seeds to uncover hidden and erroneous program behaviors, becomes faster and more effective. A lot depends on the initial seeds, which can significantly dictate the trajectory of a fuzzing campaign, particularly in terms of how long it takes to uncover interesting behaviour in your code. We introduce DIAR, a technique designed to speedup fuzzing campaigns by pinpointing and eliminating those uninteresting bytes in the seeds. Picture this: instead of wasting valuable resources on meaningless mutations in large, bloated seeds, DIAR removes the unnecessary bytes, streamlining the entire process.
In this work, we equipped AFL, a popular fuzzer, with DIAR and examined two critical Linux libraries -- Libxml's xmllint, a tool for parsing xml documents, and Binutil's readelf, an essential debugging and security analysis command-line tool used to display detailed information about ELF (Executable and Linkable Format). Our preliminary results show that AFL+DIAR does not only discover new paths more quickly but also achieves higher coverage overall. This work thus showcases how starting with lean and optimized seeds can lead to faster, more comprehensive fuzzing campaigns -- and DIAR helps you find such seeds.
- These are slides of the talk given at IEEE International Conference on Software Testing Verification and Validation Workshop, ICSTW 2022.
A tale of scale & speed: How the US Navy is enabling software delivery from l...sonjaschweigert1
Rapid and secure feature delivery is a goal across every application team and every branch of the DoD. The Navy’s DevSecOps platform, Party Barge, has achieved:
- Reduction in onboarding time from 5 weeks to 1 day
- Improved developer experience and productivity through actionable findings and reduction of false positives
- Maintenance of superior security standards and inherent policy enforcement with Authorization to Operate (ATO)
Development teams can ship efficiently and ensure applications are cyber ready for Navy Authorizing Officials (AOs). In this webinar, Sigma Defense and Anchore will give attendees a look behind the scenes and demo secure pipeline automation and security artifacts that speed up application ATO and time to production.
We will cover:
- How to remove silos in DevSecOps
- How to build efficient development pipeline roles and component templates
- How to deliver security artifacts that matter for ATO’s (SBOMs, vulnerability reports, and policy evidence)
- How to streamline operations with automated policy checks on container images
Essentials of Automations: The Art of Triggers and Actions in FMESafe Software
In this second installment of our Essentials of Automations webinar series, we’ll explore the landscape of triggers and actions, guiding you through the nuances of authoring and adapting workspaces for seamless automations. Gain an understanding of the full spectrum of triggers and actions available in FME, empowering you to enhance your workspaces for efficient automation.
We’ll kick things off by showcasing the most commonly used event-based triggers, introducing you to various automation workflows like manual triggers, schedules, directory watchers, and more. Plus, see how these elements play out in real scenarios.
Whether you’re tweaking your current setup or building from the ground up, this session will arm you with the tools and insights needed to transform your FME usage into a powerhouse of productivity. Join us to discover effective strategies that simplify complex processes, enhancing your productivity and transforming your data management practices with FME. Let’s turn complexity into clarity and make your workspaces work wonders!
Essentials of Automations: The Art of Triggers and Actions in FME
Social Data and Multimedia Analytics for News and Events Applications
1. Social Data and Multimedia Analytics for News
and Events Applications
Dr. Yiannis Kompatsiaris, ikom@iti.gr
Multimedia, Knowledge and Social Media Analytics Lab, Head
CERTH-ITI
Multimodal Social Data Management (MSDM)
Workshop
2. MSDM 2014, Athens Social Data and Multimedia Analytics#2
Overview
• Introduction
– Motivation – Challenges
• SocialSensor Project and Use Cases
• Research Approaches
– Large-Scale visual search
– Clustering
– Verification
• Demos – Applications
– MM News Demo
– Clusttour
– Thessfest
• Conclusions
3. MSDM 2014, Athens Social Data and Multimedia Analytics#3
Introduction
Motivation
Example Applications
Conceptual Architecture
Challenges
4. MSDM 2014, Athens Social Data and Multimedia Analytics
www.puzzlemarketer.com/digital-social-brands-in-60-seconds/ (Apr, 2012)
5. MSDM 2014, Athens Social Data and Multimedia Analytics
Social Networks as Real-Life Sensors
• Social Networks is a data source with an
extremely dynamic nature that reflects
events and the evolution of community
focus (user’s interests)
• Huge smartphones and mobile devices
penetration provides real-time and
location-based user feedback
• Transform individually rare but
collectively frequent media to meaningful
topics, events, points of interest,
emotional states and social connections
• Present in an efficient way for a variety of
applications (news, marketing,
entertainment)
6. MSDM 2014, Athens Social Data and Multimedia Analytics#6
Pope Francis
Pope Benedict
2007: iPhone release
2008: Android release
2010: iPad release
http://petapixel.com/2013/03/14/a-starry-sea-of-cameras-at-the-unveiling-of-pope-francis/
7. MSDM 2014, Athens Social Data and Multimedia Analytics
Social Networks as Graphs
10
social web as a graph
nodes=twi er users
edges=retweetson #jan25 hashtag
announcement of Mubarak’sresigna on
h p://gephi.org/2011/the-egyp an-revolu on-on-twi er/
8. MSDM 2014, Athens Social Data and Multimedia Analytics#8
Social Networks as Graphs
“Social networks have emergent
properties. Emergent properties
are new attributes of a whole that
arise from the interaction and
interconnection of the parts”
•Emotions, Health, Sexual
relationships do not depend just
on our connections (e.g. number
of them) but on our position -
structure in the social graph
– Central – Hub
– Outlier
– Transitivity (connections between
friends)
9. MSDM 2014, Athens Social Data and Multimedia Analytics
Examples - Science
Xin Jin, Andrew Gallagher, Liangliang Cao, Jiebo Luo, and
Jiawei Han. The wisdom of social multimedia:
using flickr for prediction and forecast,
International conference on Multimedia (MM '10). ACM.
9
“…if you're more than 100 km away from the epicenter
[of an earthquake] you can read about the quake on
twitter before it hits you…”
10. MSDM 2014, Athens Social Data and Multimedia Analytics
Example – News (Boston bombing)
#10
“Following the Boston Marathon bombings, one quarter of
Americans reportedly looked to Facebook, Twitter and
other social networking sites for information, according to
The Pew Research Center. When the Boston Police
Department posted its final “CAPTURED!!!” tweet of the
manhunt, more than 140,000 people retweeted it.”
“Authorities have recognized that one the first
places people go in events like this is to social
media, to see what the crowd is saying about what
to do next”
"I have been following my friend's
Facebook [account] who is near the scene
and she is updating everyone before it
even gets to the news”
11. MSDM 2014, Athens Social Data and Multimedia Analytics
Events - Festivals
#11
http://www.eventmanagerblog.com/uploads/2012/12/event-technology-infographic.jpg
12. MSDM 2014, Athens Social Data and Multimedia Analytics
API Wrapper
Website Wrapper
Scheduler
CRAWLING
Visual Indexing
Near-duplicates
Text Indexing
INDEXING
Media Fetcher
SNA
Sentiment - Influence
Trends - Topics
MINING
Model Building
Concepts
Relevance
Diversity
Popularity
RANKING
Veracity
Crawling Specs
Sources
Interaction
Responsiveness
Aggregation
VISUALIZATION
Aesthetics
Conceptual Architecture
13. MSDM 2014, Athens Social Data and Multimedia Analytics
Challenges – Content (Mining)
• Multi-modality: e.g. image + tags
• Rich social context: spatio-temporal, social connections,
relations and social graph
• Inconsistent quality: noise, spam, ambiguity, fake,
propaganda
• Huge volume: Massively produced and disseminated
• Multi-source: may be generated by different applications
and user communities
• Also connected to other sources (e.g. LOD, web)
• Dynamic: Fast updates, real-time
14. MSDM 2014, Athens Social Data and Multimedia Analytics
Policy – Licensing – Legal challenges
• Fragmented access to data
– Separate wrappers/APIs for each source (Twitter, Facebook, etc.)
– Different data collection/crawling policies
• Limitations imposed by API providers (“Walled Gardens”)
• Full access to data impossible or extremely expensive (e.g. see data
licensing plans for GNIP and DataSift
• Non-transparent data access practices (e.g. access is provided to an
organization/person if they have a contact in Twitter)
• Constant change of model and ToS of social APIs
– No backwards compatibility, additional development costs
• Ephemeral nature of content
• Social search results often lead to removed content inconsistent
and unreliable referencing
• User Privacy & Purpose of use
• Fuzzy regulatory framework regarding mining user-contributed data
15. MSDM 2014, Athens Social Data and Multimedia Analytics#15
Social Sensor Project
Use Cases
16. MSDM 2014, Athens Social Data and Multimedia Analytics
SocialSensor Project Objective
SocialSensor quickly surfaces trusted and relevant material
from social media – with context.
DySCODySCO
behaviou
r
location
timecontent
usage
social context
Massive social media
and unstructured web
Social media mining
Aggregation & indexing
News - Infotainment
Personalised access
Ad-hoc P2P networks
17. MSDM 2014, Athens Social Data and Multimedia Analytics#17
The SocialSensor Vision
SocialSensor quickly surfaces trusted and relevant
material from social media – with context.
•“quickly”: in real time
•“surfaces”: automatically discovers, clusters and searches
•“trusted”: automatic support in verification process
•“relevant”: to the users, personalized
•“material”: any material (text, image, audio, video =
multimedia), aggregated with other sources (e.g. web)
•“social media”: across all relevant social media platforms
•“with context”: location, time, sentiment, influence
18. MSDM 2014, Athens Social Data and Multimedia Analytics#18
Conceptual Architecture and Main components
SEMANTIC MIDDLEWARE
Public
Data
In-project
Data
SEARCH & RECOMMENDATION
USER MODELLING & PRESENTATION
INDEXINGMINING
STORAGE
DATA COLLECTION / CRAWLING
• Real time dynamic topic
and event clustering
• Trend, popularity
and sentiment analysis
• Calculate trust/influence
scores around people
• Personalized search,
access & presentation
based on social network
interactions
• Semantic enrichment
and discovery of services
19. MSDM 2014, Athens Social Data and Multimedia Analytics
Use Cases
Casual News
application
Casual News Readers
Professional
News application
Journalists, Editors, etc.
NEWS
EventLiveDashboard
Festival organizers
INFOTAINMENT
Social Media Walls
Festival attendants
20. MSDM 2014, Athens Social Data and Multimedia Analytics#20
“It has changed the way we do
news”(MSN)
“Social media is the key place for emerging stories –
internationally, nationally, locally” (BBC)
“Social media is transforming the way we do journalism”
(New York Times)
Source: picture alliance / dpa
21. MSDM 2014, Athens Social Data and Multimedia Analytics#21
Source: Getty Images
“It’s really hard to find the nuggets of useful stuff
in an ocean of content” (BBC)
“Things that aren’t relevant crowd out the content
you are looking for” (MSN)
“The filters aren’t configurable
enough” (CNN)
22. MSDM 2014, Athens Social Data and Multimedia Analytics
Verification was simpler in the past...
Source: Frank Grätz
#22
23. MSDM 2014, Athens Social Data and Multimedia Analytics#23
Infotainment
• Events with large numbers
of visitors
• Thessaloniki International
Film Festival
– 80,000 viewers / 100,000
visitors in 10 days
– 150 films, 350 screenings
• Discovery and presentation
of relevant aggregated
social media
– Trending Topics
– Sentiment
– Tweet – film matching
– Visualization (Social Walls)
24. MSDM 2014, Athens Social Data and Multimedia Analytics#24
Research Approaches
Large-Scale Visual Search
Clustering – Community Detection
Social Media Verification
25. MSDM 2014, Athens Social Data and Multimedia Analytics#25
Scalable visual feature aggregation &
indexing
• Problem: Example-based image search
– Find images that represent same or similar object or scene
with a given query image
– Viewed from different viewpoints, occlusions, clutter
• Challenge: Large-scale
– Searching databases with tens of millions of images
– Objectives to be full-filed:
• Sufficient discriminative power
• Fast response times
• Efficient memory usage
26. MSDM 2014, Athens Social Data and Multimedia Analytics#26
Large-scale visual search
image collection
from social media/
Web
image local feature
extraction
feature aggregation
feature indexingkNN visual
similarity search
concept-based
image annotation
image clustering
image (geo)tagging
concept-based
search/filtering
duplicate detection
27. MSDM 2014, Athens Social Data and Multimedia Analytics#27
Framework
• Implementation and evaluation of the effectiveness
of VLAD in combination with SURF
• Scalable image indexing
E. Spyromitros-Xioufis, et al. An Empirical Study on the
Combination of SURF Features with VLAD Vectors for Image
Search. In WIAMIS 2012, Dublin, Ireland, May 2012.
image
local
descriptor
extraction
descriptor
aggregation
dimensionality
reductionset of local
descriptors
fixed size
vector
encoding &
indexing
low dimensional
vector
SIFT / SURF BOW / VLAD PCA
PQ + ADC/IVFADC
28. MSDM 2014, Athens Social Data and Multimedia Analytics#28
Scalable indexing of features
• ADC 16x8 requires 16 bytes per image
– ~67M images per GB
• IVFADC requires 4 additional bytes per image
– ~53.6M images per GB
• In current implementation we achieve only half of above numbers due to
using short int[] instead of byte[], but possible to improve.
• Ideally, 1 billion images could be indexed on a server with
20GB of RAM (projection).
• Query time (for 1M vectors):
– Exhaustive search of VLAD vectors (d’=128): 0.50 sec
– Product Quantization with ADC 16x8: 0.10 sec (x5 faster)
– Product Quantization with IVFADC 16x8: 0.02 sec (x25 faster)
29. MSDM 2014, Athens Social Data and Multimedia Analytics#29
VLAD+SIFT vs. VLAD+SURF
Accuracy vs. dimensionality
• VLAD+SURF improves VLAD+SIFT and FV+SIFT across all dimensions in
both Holidays and Oxford datasets
Results in rows starting with * are taken from Jégou et al., 2011, hence the missing values for some entries.
SIFT corresponds to PCA reduced SIFT which yielded better results than standard SIFT in Jegou et al., 2011
30. MSDM 2014, Athens Social Data and Multimedia Analytics
Large-scale graph-based clustering
• Problem: Discover
structure in large-scale
datasets by exploiting
their relations
• Challenges - Approach:
– Large-scale
– Fast response times
– Efficient memory usage
– Noise Resilient
– Number of clusters not
known
• Structural similarity +
local expansion
community detection
techniques
31. MSDM 2014, Athens Social Data and Multimedia Analytics
• Structural similarity + Local
expansion
(highly efficient and scalable approach)
• Not necessary to know the number
of clusters
• Noise resilient
(not all nodes need to be part of a
community)
• Generic approach adaptable to
many applications
(depending on node – edge
representation)
+
S. Papadopoulos, Y. Kompatsiaris, A. Vakali. “A Graph-based Clustering Scheme for Identifying Related Tags in Folksonomies”.
In Proceedings of DaWaK'10, Springer-Verlag, 65-76
Large-scale graph-based clustering
32. MSDM 2014, Athens Social Data and Multimedia Analytics
Computational Verification in Social Media
• Create a computational verification framework to
classify tweets with unreliable media content.
• Events used for experimentation
#32
Fake images posted during Hurricane Sandy natural disaster Fake images posted during Boston Marathon bombings
33. MSDM 2014, Athens Social Data and Multimedia Analytics
Methodology
#33
34. MSDM 2014, Athens Social Data and Multimedia Analytics
Results
• Tweet Statistics
• Approaches
#34
Tweets with URLs 343939
Tweets with fake images 10758
Tweets with real images 3540
Hurricane Sandy Boston Marathon
Tweets with URLs 112449
Tweets with fake images 281
Tweets with real images 460
Classifier Classified correctly(%)
Content
features
User
features
Total
features
J48 tree 81.41 67.72 80.68
KStar 81.28 71.16 81.38
Random
Forest
80.59 70.15 80.94
Detection accuracy using cross – validation approach
Classifier Classified correctly(%)
Content
features
User
features
Total
features
J48 tree 76.45 70.81 81.25
KStar 81.28 74.12 75.78
Random
Forest
78.59 76.15 79.10
Hurricane Sandy Boston Marathon
35. MSDM 2014, Athens Social Data and Multimedia Analytics
Results(2)
#35
Classifier Classified correctly(%)
Content
features
User
features
Total
features
J48 tree 73.79 51.06 65.06
KStar 75.30 62.29 53.31
Random
Forest
74.02 63.10 65.96
Detection accuracy using different training and testing set in Hurricane Sandy
Classifier Classified correctly(%)
Content
features
User
features
Total
features
J48 tree 55.05 50.12 54.10
KStar 50.01 50.10 50.97
Random
Forest
58.75 51.03 58.78
Detection accuracy using Hurricane Sandy for training and Boston Marathon for testing
36. MSDM 2014, Athens Social Data and Multimedia Analytics#36
Other approaches
• Graph-based multimodal clustering for social event
detection in large collections of images
– automatic organization of a multimedia collection into
groups of items, each (group) of which corresponds to a
distinct event.
• Unsupervised concept learning detection using social
media as training data
• Text analysis for entities matching and sentiment
analysis
• Placing images based on content-features
• Retrieving diverse images for same entity
37. MSDM 2014, Athens Social Data and Multimedia Analytics#37
Demos - Applications
MM News Demo
Clusttour
ThesFest
38. MSDM 2014, Athens Social Data and Multimedia Analytics
Multimedia Demo
39. MSDM 2014, Athens Social Data and Multimedia Analytics#39
Multimedia Demo Architecture
#39
StreamManager
Twitter Facebook Flickr YouTube RSS Instagram
160.xx.xx.207
MongoDBWrapper
160.xx.xx.207
TextIndexer (Solr)
160.xx.xx.207
160.xx.xx.207
MediaFetcher, FeatureExtractor (HDFS)
160.xx.xx.58 160.xx.xx.107
Social Focused Crawler (HDFS)
160.xx.xx.187
Nutch
Nutch VLAD
FeatureIndexer (HDFS)
160.xx.xx.207
IVFADC
Data Mining
160.xx.xx.191
Visual Clust. Geo Clust. Statistics
Web server
160.xx.xx.116
API (3)API (4)
API (1) API (2)
40. MSDM 2014, Athens Social Data and Multimedia Analytics
tags: sagrada familia,
cathedral, barcelona
taken: 12 May 2009
lat: 41.4036, lon: 2.1743
PHOTOS & METADATA
SPATIAL CLUSTERING + TEMPORAL ANALYSIS
COMMUNITY DETECTION
CLASSIFICATION TO LANDMARKS/EVENTS
VISUAL
TAG
HYBRID
[2 years, 50 users / 120 photos]
#users / #photos
duration
[1 day, 2 users / 10 photos]
S. Papadopoulos, C. Zigkolis, Y. Kompatsiaris, A. Vakali. “Cluster-based Landmark and Event Detection on Tagged Photo
Collections”. In IEEE Multimedia Magazine 18(1), pp. 52-63, 2011
City profile creation (Clusttour)
41. MSDM 2014, Athens Social Data and Multimedia Analytics#41
City profile creation (Clusttour)
Community detection on
image similarity graphs
Nodes: photos
Edges: visual and tag
similarity
43. MSDM 2014, Athens Social Data and Multimedia Analytics#43
ThessFest
• Thessaloniki
International Film
Festival
• Support
twitter/comment
usage within the
app
• Ratings and
comments per film
• Feedback
aggregation
• Votes
• Tweets
• Real-time feedback
to the organisation
and visitors
ThessFest
44. MSDM 2014, Athens Social Data and Multimedia Analytics
Fête de la Musique Berlin app
• FETEberlin in App Store and Google Play
• More than 100K visitors
• About 5K musicians
• More than 5K app downloads, 25K
sessions
App features
•Browse and filter detailed program
•Interactive maps and routing
•Social Sharing
•Artists’ and Stages Details
•Social Monitoring
Main benefits for attendants
•Visitors can browse through maps and
don’t get lost as stages are numerous
•Event schedule is available always and per
stage
– Very useful when the server was down and
there was no access to the online schedule
#44
45. MSDM 2014, Athens Social Data and Multimedia Analytics#45
Topic analysis
• Top-10 topics
• Manual inspection
of clusters:
– 53.8% of topic titles
considered
informative
– 98.5% of clusters
were found to be
“clean”
• Topics in time
46. MSDM 2014, Athens Social Data and Multimedia Analytics
Other Application Areas
• Science
– Sociology, machine learning (machine as a teacher), computer vision
(annotation)
• Tourism – Leisure – Culture
– Off-the-beaten path POI extraction
• Marketing
– Brand monitoring, personalised ads
• Prediction
– Politics: election results
• News
– Topics, trends event detection
• Others
– Environment, emergency response, energy saving, etc
47. MSDM 2014, Athens Social Data and Multimedia Analytics
Conclusions – Further topics
• Social media data useful in many applications
• Not all data always available (e.g. User queries, fb)
– Infrastructure
– Policy - Privacy issues
• Real-time and scalable approaches
– Efficiency of semantics and analysis vs. performance vs. infrastructure
• Fusion of various modalities
– Content, social, temporal, location
• Verification & Linking other sources (web, Linked Open Data)
• Visualization - Interfaces
• Applications and commercialization
• User engagement
49. MSDM 2014, Athens Social Data and Multimedia Analytics
European Centre for Social Media
• Topics
– Social media analytics
– Verification
– Visualisation
– Applications in different domains
• Activities
– Listings of project, results, institutions, events
– Community building
– Support/organise events
– Common social media presence (e.g. LinkedIn)
– Funding from subscriptions, training, commercialisation
– Supporting projects: SocialSensor, Reveal, MULTISENSOR, PHEME,
DecarboNet, MWCC, uComp,
– Website: http://www.socialmediacentre.eu/
– Research-academic: STCSN http://stcsn.ieee.net/
50. MSDM 2014, Athens Social Data and Multimedia Analytics
Contributions from
• Dr. Symeon Papadopoulos
• Leading R&D in Social Media Mining
• Large-Scale visual search
• Community detection – Clusttour
• Dr. Sotirios Diplaris
• SocialSensor Technical Project Manager
• Lefteris Spyromitros (PhD Student, AUTH)
• Large-Scale visual search
• Christina Boididou
• Social Media Verification
• Lazaros Apostolidis
• Visualization - User Interface MM News Dem0
• Manos Schinas
• Topic Analysis
• Back-end Thessfest – Clusttour
• MM News Demo
• Juxhin Bakalli
• iOS Applications development (ThessFest - Clusttour)
• Antonis Latas
• Android Application Development (Thessfest)
51. Thank you for your attention!
ikom@iti.gr
http://mklab.iti.gr
Editor's Notes
Benefits: (i) Intelligent extraction of objects and events from the social Web, (ii) multimodal indexing and organization, (iii) personalized access and presentation of content (incl. media delivery and caching), and (iv) concrete and real integration of the social dimension of the current Web.
----- Besprechungsnotizen (03.04.12 14:41) -----
In the course of the project we have interviewed a considerable number of journalists and executives from some of the worlds biggest media outlets like CNN, the BBC, The New York Times and others... Here are some of the quotes.
3x klicken (bis alle 3 Quotes sichtbar). But journalists are not only describing the positive side. There are also huge challenges. And you can see from the following slide what is most challenging...
----- Besprechungsnotizen (03.04.12 16:44) -----
Or we can turn it the other way round: We have a known source whom the reporter trusts...
These are all candidate sources for collecting data for the system.
Ideally, we should try to have at least one source per medium (micro-blogging, photos, videos) + Facebook. Check-ins could be also valuable especially for the WP8 use case.
Partners should indicate whether they can make additional data available to the consortium.