New from BookNet Canada for 2024: BNC BiblioShare - Tech Forum 2024
Â
The WeKnowIt Project
1. Emerging, Collective Intelligence for Personal, Organisational
and Social Use
Symeon Papadopoulos,
Yiannis Kompatsiaris (CERTH-ITI)
www.weknowit.eu
Trento, April 20
ICMR 2011
3. motivation
⢠Users upload, tag, share, connect & search ď
availability of massive amounts of user-generated
content and data
⢠Existing applications are limited to simple user data
management or shallow analysis
⢠Potential for much more if we mine the data and exploit them
in the right context
4. collective intelligence
âŚa form of intelligence emerging from online user activities
Collective Intelligence >> sum of individualsâ intelligences
5. an example
one of my photos @ flickr
my location: N/A
my tags: wki experiment bcn
(âŚpretty uninformative)
6. an example
one of my photos @ flickr othersâ photos @ flickr
tags
location
7. an example
alternative views / trends / facts
one of my photos @ flickr
8. an example my friendâs photos @ flickr
one of my photos @ flickr
what did he visit next?
22. architecture / integration
Service Integration
Knowledge and Content Storage
Scenario-driven Service Composition
23. use case: emergency response
Media Intelligence
Personal Intelligence Photo arrives at ER control centre
>> Automatic localisation of photo
>> Login, Upload
>> Photo & speech auto-tagging
>> Spam detection
>> Personalized Access
Mass Intelligence
>> Clustering
>> Enrichment from additional sources
Social Intelligence
>> ER Alert Service
>> Reputation Service
Organisational Intelligence
>> Log Merging & Viewing
>> Incident Information Access
24. use case: travel Travel Preparation
Mass Intelligence
>> Landmark & Event detection
>> Ranked facet lists of POIs
>> Hybrid Image Clustering
Media Intelligence
>> Image Localisation
>> Tag suggestions Mobile Guidance
Personal Intelligence
Post Travel >> Personal Recommendations
Social Intelligence
>> Group profiling & recommendations
>> Friends position, alert
25. case: community detection in social media (1/2)
⢠Structural similarity + Local expansion
(highly efficient and scalable approach)
⢠Not necessary to know the number
of clusters
⢠Noise resilient
(not all nodes need to be part of a
+
community)
⢠Generic approach adaptable to
many applications
(depending on node â edge
representation)
S. Papadopoulos, Y. Kompatsiaris, A. Vakali. âA Graph-based Clustering Scheme for Identifying Related Tags in Folksonomiesâ.
In Proceedings of DaWaK'10, Springer-Verlag, 65-76
26. case: community detection in social media (2/2)
PHOTOS & METADATA
SPATIAL CLUSTERING + TEMPORAL ANALYSIS
tags: sagrada familia,
cathedral, barcelona
taken: 12 May 2009
lat: 41.4036, lon: 2.1743
CLASSIFICATION TO LANDMARKS/EVENTS
COMMUNITY DETECTION
VISUAL
TAG
HYBRID
S. Papadopoulos, C. Zigkolis, Y. Kompatsiaris, A. Vakali. âCluster-based Landmark and Event Detection on Tagged Photo
Collectionsâ. In IEEE Multimedia Magazine 18(1), pp. 52-63, 2011
32. conclusions
...so far
⢠CI emerges from massive online activities
⢠it is hard to extract and manage
⢠...but is definitely worth the effort.
in the future...
⢠other domains: news, finance, e-gov
⢠real-time CI
⢠CI ď Linked Data
35. content in weknowit
offline ď model creation, training
Standard annotated corpora used for training.
⢠Single-modality: text (Brown corpus), speech (TIMIT database),
standard training data image (Corel database)
⢠Single-source: prepared by a single person/organization
⢠Consistent quality: absence of spam, malicious or erroneous data
⢠Small-moderate volume: Manually produced
Massive user generated content and feedback from Web 2.0 applications
⢠Multi-modality: e.g. image + tags, image + geo-location + time
massive Web 2.0 ⢠Multi-source: may be generated by different applications, user communities,
e.g. Flickr, Panoramio, PhotoBucket
⢠Inconsistent quality: noise, spam, ambiguity
⢠Huge volume: Massively produced and disseminated
online ď user profiling, method invocation
Online content and user actions by WeKnowIt users. It is mainly used for
WKI user-contributed triggering WeKnowIt services and for providing context to them, e.g.
user profile, input content to be used as example for querying, etc.
36. technical approach
Variety of approaches depending on content-metadata input.
massive Web 2.0 â
massive Web 2.0 â standard training data semi-structured
unstructured
WKI user-contributed standard
Statistical approaches Content analysis Knowledge Based
Probabilistic models Text models (n-gram, LDA, CRF) Lookup (WordNet)
(pLSA, Bag-Of-Words) Image processing Thesaurus Lookup (GeoPlanet)
Graph-based approaches (visual feature extraction) Concept detection
(SNA, community detection) Speech modeling (Wikipedia, domain
(spectral analysis, HMM) ontologies)
37. massive Web 2.0 WKI user-contributed standard training data
Locations Topics Social connections
WP1 Get recommendations Tag normalization
Emergency alert service
Tag processing WP4
Visual analysis WP2 Community analysis tool
WP2 Text classification
Text annotation
ClustTour
Events
POI recommendation WP3 Speech search
Local tag community detector WP2
WP3 POI clustering Semantic photo query
Search place POI Entities
Named entity detection Log merger
WP5 csxPOIs WP3 WP5
Entity facet extraction - ranking Semaplorer(++)
Representation Access
CURIO Storage Account Manager
WP1 WP1
VERACITY Login
WP2 Speech Indexing
WP5 Event model F + M3O WP4 Community administration platform
WP6 Data Storage
WP6 Common data model WP5 Group Management
GUI ER CSG
WP6 Mobile app Travel preparation
Manage Item Comment
System Integration
WP1 Tag Users messaging Desktop proto WP7 Mobile guidance
WP3
Search Knowledge Base Lexical Spam Detector Post ER tool Post-travel logging
38. weknowit work structure
WP9: Management
management
WP1: Personal Intelligence
WP6: Architecture / Integration
WP2: Media Intelligence
WP3: Mass Intelligence WP7.I Use Case: ER
WP4: Social Intelligence
WP7.II Use Case: Travel
WP5: Organisational Intelligence
research development
WP8: Dissemination & Exploitation
dissemination & exploitation
39. Causality Pattern in Event-Model-F
⢠Event (cause) implies other event (effect)
⢠Causal relationship holds under some justification
⢠Causes and effects are events, and only events
40. OntoMDE
⢠Specification of MoOn using eCore and OAM as UML2 class diagram
⢠Transformation steps implemented
⢠Evaluation with ontologies of different complexity
42. results: dissemination
Activities
⢠Collective Intelligence Workshops and Special Session
⢠Summer schools
Publications
⢠8 journal publications
⢠Trans. on MultiMedia, IEEE MultiMedia, J. of Web Semantics, MTAP, etc.
⢠59 conference papers
⢠ACM MultiMedia, SIGIR, CVPR, ESWC, WWW, ICIP, WSDM, etc.
⢠2 CI book chapters + 1 CI White Paper
⢠3 patent applications