SlideShare a Scribd company logo
histoGraph 
Building a Social graph from image archives 
La science et les effets de réseau, Lyon 
24.03.2014 
www.cubrikproject.eu 
Lars Wieneke, CVCE, Luxembourg 0
The CVCE 
Lars Wieneke, CVCE, Luxembourg 1
About CUbRIK 
 European Community's Seventh Framework 
Program FP7-ICT 
 15 European partners 
 Multimedia search 
processing: Putting 
humans in the loop 
 Demos: History of Europe 
and Fashion 
Lars Wieneke, CVCE, Luxembourg 2
Point of departure 
Images as sources 
Lars Wieneke, CVCE, Luxembourg 3
Goal: Reconstructing and exploring social ties 
through historical sources 
Lars Wieneke, CVCE, Luxembourg 4
Towards the social graph 
4 pillars 
1. Close connection to the requirements of 
researchers in European Integration studies 
2. Structured and referencable repository of 
persons, events and places in time 
3. Efficient indexation process that enables the 
association of faces with identities 
4. Toolchain for analysis and visualization 
Lars Wieneke, CVCE, Luxembourg 5
Towards the social graph 
Sourcing researcher requirements 
Lars Wieneke, CVCE, Luxembourg 6 
Selection of target 
user group 
First draft of the app 
scenario 
Feedback on 
technical scope 
Exploratory 
interviews 
(daily work practices) 
Second draft of the 
app scenario 
Focus group 
(user needs and app 
scenarios) 
Feedback on 
technical feasability 
Lessons learned: 
issues and features 
Specification 
Implementation 1. 
demonstrator 
Workshop: Review of 
app and features 
Revised specification 
Implementation 2. 
demonstrator 
Evaluation and test 
Stage 1 
Stage 2 
Stage 3 
Stage 4 
Stage 5 
Users 
Requirements 
Technology
Towards the social graph 
Structured repository 
Lars Wieneke, CVCE, Luxembourg 7
Towards the social graph 
Indexation process 
8 
Raw content 
High level features 
(automatic annotations) 
Conflict 
(e.g., “Image contains 
‘Romano Prodi’ ” 
? Confidence = low) 
Conflict store Conflict 
manager 
Conflict resolution 
task store 
Conflict resolution 
task: conflict, 
required skill, priority, .. 
CUbRIK app 
for Conflict 
resolution 
Game Crowdtask Q&A 
Lars Wieneke, CVCE, Luxembourg
9 
Towards the social graph 
Indexation process II 
 Human in the loop added value: 
 Verification of identities/places/events ambiguous and temporal only 
possible by putting humans in the loop 
 Integration of multiple perspectives 
 CUbRIK as an open toolbox allows 
follow-up and extension through 
third parties 
 “Vertical” integration: 
GUI, components, crowdsourcing 
integrated in a platform 
Lars Wieneke, CVCE, Luxembourg
10 
Towards the social graph 
Visualization and analysis 
Lars Wieneke, CVCE, Luxembourg
11 
Challenges & Approach 
 Main challenges 
 Detection and identification of identities/places/events in time 
 Verification of identities/places/events in time 
 Analysis of relationships (e.g. co-occurrences) 
 Rights aware crawling and storage 
 Verification of provenance and license information 
 Truth and provenance 
 Approach 
 Crowd-sourced verification of detected faces (false positives/negatives) 
 Verification of identities through/places/events in time social networks of 
experts 
 Visual knowledge discovery/exploration 
 Integrated rights aware crawling and storage 
 Integrated license and provenance management 
27/11/2013 Lars Wieneke, CVCE, Luxembourg
Towards the social graph 
Bringing it all together 
Social Graph Network Analysis 
CROWD 
Research 
Inquiry 
Graph 
Visualizat ion 
Lars Wieneke, CVCE, Luxembourg 12 
Image Indexation 
Media 
harvest ing 
and upload 
Face 
detect ion 
Face 
ident ifi cat ion 
Clickworkers 
Crowd Face 
posit ion 
validat ion 
Copyright 
aware 
crawler 
Provenance 
checker 
License 
checker 
Content 
provider 
tools 
Metadata 
Ent ity 
extract ion 
Identity reconciliation 
Ent ity 
verifi cat ion & 
annotat ion 
Ent itypedia 
Integrat ion 
CROWD 
pre-fi 
ltering 
Text Indexation 
Connect ion 
to the CVCE 
collect ion 
Ent ity 
anntat ion and 
extract ion 
Expert Crowd 
Expert 
CROWD 
verifi cat ion 
Ent itypedia 
Integrat ion 
CROWD Research Inquieries 
Expert Crowd 
Graph 
Visualizat ion 
Analysis of 
the social 
graph 
Graph Query (old: Query for Entities) 
Graph 
Visualizat ion 
Query for 
ent it ies 
Context Expander 
Expansion 
through 
documents 
Expansion 
through 
videos 
Expansion 
through 
images 
Expansion 
through 
related 
ent it ies 
Social Graph construction 
Social 
Graph 
Creat ion 
Content Analysis and 
Enrichment 
Querying 
Feedback acquisition 
and processing 
Y3 component 
Query for 
spat ial 
constraints 
EXP through 
SIMILAR 
images 
WP Event 
detect ion
histoGraph demo 
Time for a Demo! 
Lars Wieneke, CVCE, Luxembourg 13
Pipelining the CUbRIK components: 
Human input from click-workers 
Great choice for simple tasks: 
 Face detection: false positives, false negatives 
 Monetary motivation, via www.microtask.com 
Poor performance on complex tasks: 
 Low resolution images 
 Different angles etc. 
 Actors recurring over time 
Lars Wieneke, CVCE, Luxembourg 14
Pipelining the CUbRIK components: 
Human input from experts 
Capable of complex tasks: 
 In-depth knowledge of key actors 
 Context knowledge allows inferences 
But: Different motivational models! 
 Public goods 
 Reputation 
Lars Wieneke, CVCE, Luxembourg 15
Usage for historians 
 No one truth in history but interpretation, 
context and discussion 
 Therefore need to represent ambivalence, 
contradictions and discussion 
 Close ties between data representation (Social 
graph) and their original context (primary 
sources) 
Lars Wieneke, CVCE, Luxembourg 16
Conclusion 
 Challenges 
 What is truth? Humanities vs. Computer Science 
 Gathering requirements for tools that haven‘t been 
developed yet 
 Engaging crowds 
 Image copyrights 
 Scientific value? 
 Refinement of the application 
 Additional datasources 
 Improvement of the interface 
 Integration of the new components 
Lars Wieneke, CVCE, Luxembourg 17
Outlook 
CUbRIK Presentation 18
Outlook 
CUbRIK Presentation 19
Outlook histoGraph 
CUbRIK Presentation 20
Visit us on 
WWW.CUBRIKPROJECT.EU 
@CUBRIKPROJECT 
CUbRIK Presentation 21 
Or follow us on Twitter

More Related Content

Viewers also liked

CUbRIK Summer School RHodes histoGraph
CUbRIK Summer School RHodes histoGraphCUbRIK Summer School RHodes histoGraph
CUbRIK Summer School RHodes histoGraphdhlab
 
History of Europe demo at IEEE MMSP 2013
History of Europe demo at IEEE MMSP 2013History of Europe demo at IEEE MMSP 2013
History of Europe demo at IEEE MMSP 2013dhlab
 
Algebra de baldor by. aimb
Algebra de baldor by. aimbAlgebra de baldor by. aimb
Algebra de baldor by. aimbAlex Mindiola
 
Europe’s Beginnings through the Looking Glass: Publishing Historical Document...
Europe’s Beginnings through the Looking Glass: Publishing Historical Document...Europe’s Beginnings through the Looking Glass: Publishing Historical Document...
Europe’s Beginnings through the Looking Glass: Publishing Historical Document...dhlab
 
DH2013: Lars Wieneke – Workshop introduction
DH2013: Lars Wieneke – Workshop introduction DH2013: Lars Wieneke – Workshop introduction
DH2013: Lars Wieneke – Workshop introduction dhlab
 
DH2013: Christine Sauter – Results of the task force
DH2013: Christine Sauter – Results of the task forceDH2013: Christine Sauter – Results of the task force
DH2013: Christine Sauter – Results of the task forcedhlab
 
DH2013: Stuart Dunn - An emerging field(?): defining the fundamentals of huma...
DH2013: Stuart Dunn - An emerging field(?): defining the fundamentals of huma...DH2013: Stuart Dunn - An emerging field(?): defining the fundamentals of huma...
DH2013: Stuart Dunn - An emerging field(?): defining the fundamentals of huma...dhlab
 
DH2013: Marion Dupeyrat – Interacting with audiences: overview of participato...
DH2013: Marion Dupeyrat – Interacting with audiences: overview of participato...DH2013: Marion Dupeyrat – Interacting with audiences: overview of participato...
DH2013: Marion Dupeyrat – Interacting with audiences: overview of participato...dhlab
 
DH2013: Ad Pollé – Europeana 1914-18 & Europeana 1989
DH2013: Ad Pollé – Europeana 1914-18 & Europeana 1989DH2013: Ad Pollé – Europeana 1914-18 & Europeana 1989
DH2013: Ad Pollé – Europeana 1914-18 & Europeana 1989dhlab
 
DH2013: Julia Fallon – Legal aspects of UGC
DH2013: Julia Fallon – Legal aspects of UGCDH2013: Julia Fallon – Legal aspects of UGC
DH2013: Julia Fallon – Legal aspects of UGCdhlab
 
Text Encoding and Enrichment for Linguistic Analysis: Archives on the policy ...
Text Encoding and Enrichment for Linguistic Analysis: Archives on the policy ...Text Encoding and Enrichment for Linguistic Analysis: Archives on the policy ...
Text Encoding and Enrichment for Linguistic Analysis: Archives on the policy ...dhlab
 
MyPublications: Enabling personal authoring and narrative making
MyPublications: Enabling personal authoring and narrative makingMyPublications: Enabling personal authoring and narrative making
MyPublications: Enabling personal authoring and narrative makingdhlab
 
KIK Skpp8(1) INNOTECH-S
KIK Skpp8(1) INNOTECH-SKIK Skpp8(1) INNOTECH-S
KIK Skpp8(1) INNOTECH-Smengtm
 

Viewers also liked (14)

CUbRIK Summer School RHodes histoGraph
CUbRIK Summer School RHodes histoGraphCUbRIK Summer School RHodes histoGraph
CUbRIK Summer School RHodes histoGraph
 
History of Europe demo at IEEE MMSP 2013
History of Europe demo at IEEE MMSP 2013History of Europe demo at IEEE MMSP 2013
History of Europe demo at IEEE MMSP 2013
 
Algebra de baldor by. aimb
Algebra de baldor by. aimbAlgebra de baldor by. aimb
Algebra de baldor by. aimb
 
Europe’s Beginnings through the Looking Glass: Publishing Historical Document...
Europe’s Beginnings through the Looking Glass: Publishing Historical Document...Europe’s Beginnings through the Looking Glass: Publishing Historical Document...
Europe’s Beginnings through the Looking Glass: Publishing Historical Document...
 
DH2013: Lars Wieneke – Workshop introduction
DH2013: Lars Wieneke – Workshop introduction DH2013: Lars Wieneke – Workshop introduction
DH2013: Lars Wieneke – Workshop introduction
 
DH2013: Christine Sauter – Results of the task force
DH2013: Christine Sauter – Results of the task forceDH2013: Christine Sauter – Results of the task force
DH2013: Christine Sauter – Results of the task force
 
DH2013: Stuart Dunn - An emerging field(?): defining the fundamentals of huma...
DH2013: Stuart Dunn - An emerging field(?): defining the fundamentals of huma...DH2013: Stuart Dunn - An emerging field(?): defining the fundamentals of huma...
DH2013: Stuart Dunn - An emerging field(?): defining the fundamentals of huma...
 
DH2013: Marion Dupeyrat – Interacting with audiences: overview of participato...
DH2013: Marion Dupeyrat – Interacting with audiences: overview of participato...DH2013: Marion Dupeyrat – Interacting with audiences: overview of participato...
DH2013: Marion Dupeyrat – Interacting with audiences: overview of participato...
 
DH2013: Ad Pollé – Europeana 1914-18 & Europeana 1989
DH2013: Ad Pollé – Europeana 1914-18 & Europeana 1989DH2013: Ad Pollé – Europeana 1914-18 & Europeana 1989
DH2013: Ad Pollé – Europeana 1914-18 & Europeana 1989
 
DH2013: Julia Fallon – Legal aspects of UGC
DH2013: Julia Fallon – Legal aspects of UGCDH2013: Julia Fallon – Legal aspects of UGC
DH2013: Julia Fallon – Legal aspects of UGC
 
Text Encoding and Enrichment for Linguistic Analysis: Archives on the policy ...
Text Encoding and Enrichment for Linguistic Analysis: Archives on the policy ...Text Encoding and Enrichment for Linguistic Analysis: Archives on the policy ...
Text Encoding and Enrichment for Linguistic Analysis: Archives on the policy ...
 
MyPublications: Enabling personal authoring and narrative making
MyPublications: Enabling personal authoring and narrative makingMyPublications: Enabling personal authoring and narrative making
MyPublications: Enabling personal authoring and narrative making
 
KIK Skpp8(1) INNOTECH-S
KIK Skpp8(1) INNOTECH-SKIK Skpp8(1) INNOTECH-S
KIK Skpp8(1) INNOTECH-S
 
майстер клас
майстер клас майстер клас
майстер клас
 

Similar to HistoGraph presentation Insa de Lyon

histoGraph for historians
histoGraph for historianshistoGraph for historians
histoGraph for historiansCUbRIK Project
 
Humanist machine interaction with histoGraph
Humanist machine interaction with histoGraphHumanist machine interaction with histoGraph
Humanist machine interaction with histoGraphCUbRIK Project
 
From Argument Mapping to Argument Mining, and Back
From Argument Mapping to Argument Mining, and BackFrom Argument Mapping to Argument Mining, and Back
From Argument Mapping to Argument Mining, and BackEDV Project
 
(Industry track) "Interactive networks for digital cultural heritage collecti...
(Industry track) "Interactive networks for digital cultural heritage collecti...(Industry track) "Interactive networks for digital cultural heritage collecti...
(Industry track) "Interactive networks for digital cultural heritage collecti...icwe2015
 
SocialSensor Project: Sensing User Generated Input for Improved Media Discove...
SocialSensor Project: Sensing User Generated Input for Improved Media Discove...SocialSensor Project: Sensing User Generated Input for Improved Media Discove...
SocialSensor Project: Sensing User Generated Input for Improved Media Discove...Yiannis Kompatsiaris
 
ICQE20: Quantitative Ethnography Visualizations as Tools for Thinking
ICQE20: Quantitative Ethnography Visualizations as Tools for ThinkingICQE20: Quantitative Ethnography Visualizations as Tools for Thinking
ICQE20: Quantitative Ethnography Visualizations as Tools for ThinkingSimon Buckingham Shum
 
Social Media Metrics for the Cultural Heritage sector
Social Media Metrics for the Cultural Heritage sectorSocial Media Metrics for the Cultural Heritage sector
Social Media Metrics for the Cultural Heritage sectorHU-Crossmedialab
 
Multimedia Information Retrieval: Bytes and pixels meet the challenges of hum...
Multimedia Information Retrieval: Bytes and pixels meet the challenges of hum...Multimedia Information Retrieval: Bytes and pixels meet the challenges of hum...
Multimedia Information Retrieval: Bytes and pixels meet the challenges of hum...maranlar
 
TTO Keynote 08 10 2021
TTO Keynote 08 10 2021TTO Keynote 08 10 2021
TTO Keynote 08 10 2021Weverify
 
Survey of Semantic Media Annotation Tools - towards New Media Applications wi...
Survey of Semantic Media Annotation Tools - towards New Media Applications wi...Survey of Semantic Media Annotation Tools - towards New Media Applications wi...
Survey of Semantic Media Annotation Tools - towards New Media Applications wi...LinkedTV
 
Science of culture? Computational analysis and visualization of cultural imag...
Science of culture? Computational analysis and visualization of cultural imag...Science of culture? Computational analysis and visualization of cultural imag...
Science of culture? Computational analysis and visualization of cultural imag...Lev Manovich
 
Building a social graph for the history of Europe: the CUbRIK histoGraph
Building a social graph for the history of Europe: the CUbRIK histoGraphBuilding a social graph for the history of Europe: the CUbRIK histoGraph
Building a social graph for the history of Europe: the CUbRIK histoGraphCUbRIK Project
 
Visualization for Software Analytics
Visualization for Software AnalyticsVisualization for Software Analytics
Visualization for Software AnalyticsMargaret-Anne Storey
 
SMU MBA Solved Assignment MK0016
 SMU  MBA Solved Assignment MK0016 SMU  MBA Solved Assignment MK0016
SMU MBA Solved Assignment MK0016Revlon
 
Social Requirements Engineering and the RWTH Aachen University Test Bed
Social Requirements Engineering and the RWTH Aachen University Test BedSocial Requirements Engineering and the RWTH Aachen University Test Bed
Social Requirements Engineering and the RWTH Aachen University Test BedRalf Klamma
 
Smart City R&D Activities in Vienna
Smart City R&D Activities in ViennaSmart City R&D Activities in Vienna
Smart City R&D Activities in ViennaJIC
 
Linked Data in Production: Moving Beyond Ontologies
Linked Data in Production: Moving Beyond OntologiesLinked Data in Production: Moving Beyond Ontologies
Linked Data in Production: Moving Beyond OntologiesDavid Newbury
 
WSI Stimulus Project: Centre for longitudinal studies of online citizen parti...
WSI Stimulus Project: Centre for longitudinal studies of online citizen parti...WSI Stimulus Project: Centre for longitudinal studies of online citizen parti...
WSI Stimulus Project: Centre for longitudinal studies of online citizen parti...Ramine Tinati
 

Similar to HistoGraph presentation Insa de Lyon (20)

histoGraph for historians
histoGraph for historianshistoGraph for historians
histoGraph for historians
 
Humanist machine interaction with histoGraph
Humanist machine interaction with histoGraphHumanist machine interaction with histoGraph
Humanist machine interaction with histoGraph
 
From Argument Mapping to Argument Mining, and Back
From Argument Mapping to Argument Mining, and BackFrom Argument Mapping to Argument Mining, and Back
From Argument Mapping to Argument Mining, and Back
 
(Industry track) "Interactive networks for digital cultural heritage collecti...
(Industry track) "Interactive networks for digital cultural heritage collecti...(Industry track) "Interactive networks for digital cultural heritage collecti...
(Industry track) "Interactive networks for digital cultural heritage collecti...
 
SocialSensor Project: Sensing User Generated Input for Improved Media Discove...
SocialSensor Project: Sensing User Generated Input for Improved Media Discove...SocialSensor Project: Sensing User Generated Input for Improved Media Discove...
SocialSensor Project: Sensing User Generated Input for Improved Media Discove...
 
Rogers digitalmethods 4nov2010
Rogers digitalmethods 4nov2010Rogers digitalmethods 4nov2010
Rogers digitalmethods 4nov2010
 
ICQE20: Quantitative Ethnography Visualizations as Tools for Thinking
ICQE20: Quantitative Ethnography Visualizations as Tools for ThinkingICQE20: Quantitative Ethnography Visualizations as Tools for Thinking
ICQE20: Quantitative Ethnography Visualizations as Tools for Thinking
 
Social Media Metrics for the Cultural Heritage sector
Social Media Metrics for the Cultural Heritage sectorSocial Media Metrics for the Cultural Heritage sector
Social Media Metrics for the Cultural Heritage sector
 
Multimedia Information Retrieval: Bytes and pixels meet the challenges of hum...
Multimedia Information Retrieval: Bytes and pixels meet the challenges of hum...Multimedia Information Retrieval: Bytes and pixels meet the challenges of hum...
Multimedia Information Retrieval: Bytes and pixels meet the challenges of hum...
 
TTO Keynote 08 10 2021
TTO Keynote 08 10 2021TTO Keynote 08 10 2021
TTO Keynote 08 10 2021
 
Survey of Semantic Media Annotation Tools - towards New Media Applications wi...
Survey of Semantic Media Annotation Tools - towards New Media Applications wi...Survey of Semantic Media Annotation Tools - towards New Media Applications wi...
Survey of Semantic Media Annotation Tools - towards New Media Applications wi...
 
Science of culture? Computational analysis and visualization of cultural imag...
Science of culture? Computational analysis and visualization of cultural imag...Science of culture? Computational analysis and visualization of cultural imag...
Science of culture? Computational analysis and visualization of cultural imag...
 
Building a social graph for the history of Europe: the CUbRIK histoGraph
Building a social graph for the history of Europe: the CUbRIK histoGraphBuilding a social graph for the history of Europe: the CUbRIK histoGraph
Building a social graph for the history of Europe: the CUbRIK histoGraph
 
Visualization for Software Analytics
Visualization for Software AnalyticsVisualization for Software Analytics
Visualization for Software Analytics
 
SMU MBA Solved Assignment MK0016
 SMU  MBA Solved Assignment MK0016 SMU  MBA Solved Assignment MK0016
SMU MBA Solved Assignment MK0016
 
Social Requirements Engineering and the RWTH Aachen University Test Bed
Social Requirements Engineering and the RWTH Aachen University Test BedSocial Requirements Engineering and the RWTH Aachen University Test Bed
Social Requirements Engineering and the RWTH Aachen University Test Bed
 
Long uglytestingdeck
Long uglytestingdeckLong uglytestingdeck
Long uglytestingdeck
 
Smart City R&D Activities in Vienna
Smart City R&D Activities in ViennaSmart City R&D Activities in Vienna
Smart City R&D Activities in Vienna
 
Linked Data in Production: Moving Beyond Ontologies
Linked Data in Production: Moving Beyond OntologiesLinked Data in Production: Moving Beyond Ontologies
Linked Data in Production: Moving Beyond Ontologies
 
WSI Stimulus Project: Centre for longitudinal studies of online citizen parti...
WSI Stimulus Project: Centre for longitudinal studies of online citizen parti...WSI Stimulus Project: Centre for longitudinal studies of online citizen parti...
WSI Stimulus Project: Centre for longitudinal studies of online citizen parti...
 

Recently uploaded

Hemoglobin metabolism: C Kalyan & E. Muralinath
Hemoglobin metabolism: C Kalyan & E. MuralinathHemoglobin metabolism: C Kalyan & E. Muralinath
Hemoglobin metabolism: C Kalyan & E. Muralinathmuralinath2
 
Pests of sugarcane_Binomics_IPM_Dr.UPR.pdf
Pests of sugarcane_Binomics_IPM_Dr.UPR.pdfPests of sugarcane_Binomics_IPM_Dr.UPR.pdf
Pests of sugarcane_Binomics_IPM_Dr.UPR.pdfPirithiRaju
 
Detectability of Solar Panels as a Technosignature
Detectability of Solar Panels as a TechnosignatureDetectability of Solar Panels as a Technosignature
Detectability of Solar Panels as a TechnosignatureSérgio Sacani
 
PLANT DISEASE MANAGEMENT PRINCIPLES AND ITS IMPORTANCE
PLANT DISEASE MANAGEMENT PRINCIPLES AND ITS IMPORTANCEPLANT DISEASE MANAGEMENT PRINCIPLES AND ITS IMPORTANCE
PLANT DISEASE MANAGEMENT PRINCIPLES AND ITS IMPORTANCETALAPATI ARUNA CHENNA VYDYANAD
 
NuGOweek 2024 full programme - hosted by Ghent University
NuGOweek 2024 full programme - hosted by Ghent UniversityNuGOweek 2024 full programme - hosted by Ghent University
NuGOweek 2024 full programme - hosted by Ghent Universitypablovgd
 
A Giant Impact Origin for the First Subduction on Earth
A Giant Impact Origin for the First Subduction on EarthA Giant Impact Origin for the First Subduction on Earth
A Giant Impact Origin for the First Subduction on EarthSérgio Sacani
 
Triploidy ...............................pptx
Triploidy ...............................pptxTriploidy ...............................pptx
Triploidy ...............................pptxCherry
 
GBSN - Microbiology Lab 2 (Compound Microscope)
GBSN - Microbiology Lab 2 (Compound Microscope)GBSN - Microbiology Lab 2 (Compound Microscope)
GBSN - Microbiology Lab 2 (Compound Microscope)Areesha Ahmad
 
Gliese 12 b, a temperate Earth-sized planet at 12 parsecs discovered with TES...
Gliese 12 b, a temperate Earth-sized planet at 12 parsecs discovered with TES...Gliese 12 b, a temperate Earth-sized planet at 12 parsecs discovered with TES...
Gliese 12 b, a temperate Earth-sized planet at 12 parsecs discovered with TES...Sérgio Sacani
 
Aerodynamics. flippatterncn5tm5ttnj6nmnynyppt
Aerodynamics. flippatterncn5tm5ttnj6nmnynypptAerodynamics. flippatterncn5tm5ttnj6nmnynyppt
Aerodynamics. flippatterncn5tm5ttnj6nmnynypptsreddyrahul
 
Topography and sediments of the floor of the Bay of Bengal
Topography and sediments of the floor of the Bay of BengalTopography and sediments of the floor of the Bay of Bengal
Topography and sediments of the floor of the Bay of BengalMd Hasan Tareq
 
Structural annotation................pptx
Structural annotation................pptxStructural annotation................pptx
Structural annotation................pptxCherry
 
The importance of continents, oceans and plate tectonics for the evolution of...
The importance of continents, oceans and plate tectonics for the evolution of...The importance of continents, oceans and plate tectonics for the evolution of...
The importance of continents, oceans and plate tectonics for the evolution of...Sérgio Sacani
 
GBSN - Microbiology (Unit 6) Human and Microbial interaction
GBSN - Microbiology (Unit 6) Human and Microbial interactionGBSN - Microbiology (Unit 6) Human and Microbial interaction
GBSN - Microbiology (Unit 6) Human and Microbial interactionAreesha Ahmad
 
GBSN - Biochemistry (Unit 4) Chemistry of Carbohydrates
GBSN - Biochemistry (Unit 4) Chemistry of CarbohydratesGBSN - Biochemistry (Unit 4) Chemistry of Carbohydrates
GBSN - Biochemistry (Unit 4) Chemistry of CarbohydratesAreesha Ahmad
 
Isolation of AMF by wet sieving and decantation method pptx
Isolation of AMF by wet sieving and decantation method pptxIsolation of AMF by wet sieving and decantation method pptx
Isolation of AMF by wet sieving and decantation method pptxGOWTHAMIM22
 
Ostiguy & Panizza & Moffitt (eds.) - Populism in Global Perspective. A Perfor...
Ostiguy & Panizza & Moffitt (eds.) - Populism in Global Perspective. A Perfor...Ostiguy & Panizza & Moffitt (eds.) - Populism in Global Perspective. A Perfor...
Ostiguy & Panizza & Moffitt (eds.) - Populism in Global Perspective. A Perfor...frank0071
 
Pests of Green Manures_Bionomics_IPM_Dr.UPR.pdf
Pests of Green Manures_Bionomics_IPM_Dr.UPR.pdfPests of Green Manures_Bionomics_IPM_Dr.UPR.pdf
Pests of Green Manures_Bionomics_IPM_Dr.UPR.pdfPirithiRaju
 
mixotrophy in cyanobacteria: a dual nutritional strategy
mixotrophy in cyanobacteria: a dual nutritional strategymixotrophy in cyanobacteria: a dual nutritional strategy
mixotrophy in cyanobacteria: a dual nutritional strategyMansiBishnoi1
 
B lymphocytes, Receptors, Maturation and Activation
B lymphocytes, Receptors, Maturation and ActivationB lymphocytes, Receptors, Maturation and Activation
B lymphocytes, Receptors, Maturation and ActivationBhanu Krishan
 

Recently uploaded (20)

Hemoglobin metabolism: C Kalyan & E. Muralinath
Hemoglobin metabolism: C Kalyan & E. MuralinathHemoglobin metabolism: C Kalyan & E. Muralinath
Hemoglobin metabolism: C Kalyan & E. Muralinath
 
Pests of sugarcane_Binomics_IPM_Dr.UPR.pdf
Pests of sugarcane_Binomics_IPM_Dr.UPR.pdfPests of sugarcane_Binomics_IPM_Dr.UPR.pdf
Pests of sugarcane_Binomics_IPM_Dr.UPR.pdf
 
Detectability of Solar Panels as a Technosignature
Detectability of Solar Panels as a TechnosignatureDetectability of Solar Panels as a Technosignature
Detectability of Solar Panels as a Technosignature
 
PLANT DISEASE MANAGEMENT PRINCIPLES AND ITS IMPORTANCE
PLANT DISEASE MANAGEMENT PRINCIPLES AND ITS IMPORTANCEPLANT DISEASE MANAGEMENT PRINCIPLES AND ITS IMPORTANCE
PLANT DISEASE MANAGEMENT PRINCIPLES AND ITS IMPORTANCE
 
NuGOweek 2024 full programme - hosted by Ghent University
NuGOweek 2024 full programme - hosted by Ghent UniversityNuGOweek 2024 full programme - hosted by Ghent University
NuGOweek 2024 full programme - hosted by Ghent University
 
A Giant Impact Origin for the First Subduction on Earth
A Giant Impact Origin for the First Subduction on EarthA Giant Impact Origin for the First Subduction on Earth
A Giant Impact Origin for the First Subduction on Earth
 
Triploidy ...............................pptx
Triploidy ...............................pptxTriploidy ...............................pptx
Triploidy ...............................pptx
 
GBSN - Microbiology Lab 2 (Compound Microscope)
GBSN - Microbiology Lab 2 (Compound Microscope)GBSN - Microbiology Lab 2 (Compound Microscope)
GBSN - Microbiology Lab 2 (Compound Microscope)
 
Gliese 12 b, a temperate Earth-sized planet at 12 parsecs discovered with TES...
Gliese 12 b, a temperate Earth-sized planet at 12 parsecs discovered with TES...Gliese 12 b, a temperate Earth-sized planet at 12 parsecs discovered with TES...
Gliese 12 b, a temperate Earth-sized planet at 12 parsecs discovered with TES...
 
Aerodynamics. flippatterncn5tm5ttnj6nmnynyppt
Aerodynamics. flippatterncn5tm5ttnj6nmnynypptAerodynamics. flippatterncn5tm5ttnj6nmnynyppt
Aerodynamics. flippatterncn5tm5ttnj6nmnynyppt
 
Topography and sediments of the floor of the Bay of Bengal
Topography and sediments of the floor of the Bay of BengalTopography and sediments of the floor of the Bay of Bengal
Topography and sediments of the floor of the Bay of Bengal
 
Structural annotation................pptx
Structural annotation................pptxStructural annotation................pptx
Structural annotation................pptx
 
The importance of continents, oceans and plate tectonics for the evolution of...
The importance of continents, oceans and plate tectonics for the evolution of...The importance of continents, oceans and plate tectonics for the evolution of...
The importance of continents, oceans and plate tectonics for the evolution of...
 
GBSN - Microbiology (Unit 6) Human and Microbial interaction
GBSN - Microbiology (Unit 6) Human and Microbial interactionGBSN - Microbiology (Unit 6) Human and Microbial interaction
GBSN - Microbiology (Unit 6) Human and Microbial interaction
 
GBSN - Biochemistry (Unit 4) Chemistry of Carbohydrates
GBSN - Biochemistry (Unit 4) Chemistry of CarbohydratesGBSN - Biochemistry (Unit 4) Chemistry of Carbohydrates
GBSN - Biochemistry (Unit 4) Chemistry of Carbohydrates
 
Isolation of AMF by wet sieving and decantation method pptx
Isolation of AMF by wet sieving and decantation method pptxIsolation of AMF by wet sieving and decantation method pptx
Isolation of AMF by wet sieving and decantation method pptx
 
Ostiguy & Panizza & Moffitt (eds.) - Populism in Global Perspective. A Perfor...
Ostiguy & Panizza & Moffitt (eds.) - Populism in Global Perspective. A Perfor...Ostiguy & Panizza & Moffitt (eds.) - Populism in Global Perspective. A Perfor...
Ostiguy & Panizza & Moffitt (eds.) - Populism in Global Perspective. A Perfor...
 
Pests of Green Manures_Bionomics_IPM_Dr.UPR.pdf
Pests of Green Manures_Bionomics_IPM_Dr.UPR.pdfPests of Green Manures_Bionomics_IPM_Dr.UPR.pdf
Pests of Green Manures_Bionomics_IPM_Dr.UPR.pdf
 
mixotrophy in cyanobacteria: a dual nutritional strategy
mixotrophy in cyanobacteria: a dual nutritional strategymixotrophy in cyanobacteria: a dual nutritional strategy
mixotrophy in cyanobacteria: a dual nutritional strategy
 
B lymphocytes, Receptors, Maturation and Activation
B lymphocytes, Receptors, Maturation and ActivationB lymphocytes, Receptors, Maturation and Activation
B lymphocytes, Receptors, Maturation and Activation
 

HistoGraph presentation Insa de Lyon

  • 1. histoGraph Building a Social graph from image archives La science et les effets de réseau, Lyon 24.03.2014 www.cubrikproject.eu Lars Wieneke, CVCE, Luxembourg 0
  • 2. The CVCE Lars Wieneke, CVCE, Luxembourg 1
  • 3. About CUbRIK  European Community's Seventh Framework Program FP7-ICT  15 European partners  Multimedia search processing: Putting humans in the loop  Demos: History of Europe and Fashion Lars Wieneke, CVCE, Luxembourg 2
  • 4. Point of departure Images as sources Lars Wieneke, CVCE, Luxembourg 3
  • 5. Goal: Reconstructing and exploring social ties through historical sources Lars Wieneke, CVCE, Luxembourg 4
  • 6. Towards the social graph 4 pillars 1. Close connection to the requirements of researchers in European Integration studies 2. Structured and referencable repository of persons, events and places in time 3. Efficient indexation process that enables the association of faces with identities 4. Toolchain for analysis and visualization Lars Wieneke, CVCE, Luxembourg 5
  • 7. Towards the social graph Sourcing researcher requirements Lars Wieneke, CVCE, Luxembourg 6 Selection of target user group First draft of the app scenario Feedback on technical scope Exploratory interviews (daily work practices) Second draft of the app scenario Focus group (user needs and app scenarios) Feedback on technical feasability Lessons learned: issues and features Specification Implementation 1. demonstrator Workshop: Review of app and features Revised specification Implementation 2. demonstrator Evaluation and test Stage 1 Stage 2 Stage 3 Stage 4 Stage 5 Users Requirements Technology
  • 8. Towards the social graph Structured repository Lars Wieneke, CVCE, Luxembourg 7
  • 9. Towards the social graph Indexation process 8 Raw content High level features (automatic annotations) Conflict (e.g., “Image contains ‘Romano Prodi’ ” ? Confidence = low) Conflict store Conflict manager Conflict resolution task store Conflict resolution task: conflict, required skill, priority, .. CUbRIK app for Conflict resolution Game Crowdtask Q&A Lars Wieneke, CVCE, Luxembourg
  • 10. 9 Towards the social graph Indexation process II  Human in the loop added value:  Verification of identities/places/events ambiguous and temporal only possible by putting humans in the loop  Integration of multiple perspectives  CUbRIK as an open toolbox allows follow-up and extension through third parties  “Vertical” integration: GUI, components, crowdsourcing integrated in a platform Lars Wieneke, CVCE, Luxembourg
  • 11. 10 Towards the social graph Visualization and analysis Lars Wieneke, CVCE, Luxembourg
  • 12. 11 Challenges & Approach  Main challenges  Detection and identification of identities/places/events in time  Verification of identities/places/events in time  Analysis of relationships (e.g. co-occurrences)  Rights aware crawling and storage  Verification of provenance and license information  Truth and provenance  Approach  Crowd-sourced verification of detected faces (false positives/negatives)  Verification of identities through/places/events in time social networks of experts  Visual knowledge discovery/exploration  Integrated rights aware crawling and storage  Integrated license and provenance management 27/11/2013 Lars Wieneke, CVCE, Luxembourg
  • 13. Towards the social graph Bringing it all together Social Graph Network Analysis CROWD Research Inquiry Graph Visualizat ion Lars Wieneke, CVCE, Luxembourg 12 Image Indexation Media harvest ing and upload Face detect ion Face ident ifi cat ion Clickworkers Crowd Face posit ion validat ion Copyright aware crawler Provenance checker License checker Content provider tools Metadata Ent ity extract ion Identity reconciliation Ent ity verifi cat ion & annotat ion Ent itypedia Integrat ion CROWD pre-fi ltering Text Indexation Connect ion to the CVCE collect ion Ent ity anntat ion and extract ion Expert Crowd Expert CROWD verifi cat ion Ent itypedia Integrat ion CROWD Research Inquieries Expert Crowd Graph Visualizat ion Analysis of the social graph Graph Query (old: Query for Entities) Graph Visualizat ion Query for ent it ies Context Expander Expansion through documents Expansion through videos Expansion through images Expansion through related ent it ies Social Graph construction Social Graph Creat ion Content Analysis and Enrichment Querying Feedback acquisition and processing Y3 component Query for spat ial constraints EXP through SIMILAR images WP Event detect ion
  • 14. histoGraph demo Time for a Demo! Lars Wieneke, CVCE, Luxembourg 13
  • 15. Pipelining the CUbRIK components: Human input from click-workers Great choice for simple tasks:  Face detection: false positives, false negatives  Monetary motivation, via www.microtask.com Poor performance on complex tasks:  Low resolution images  Different angles etc.  Actors recurring over time Lars Wieneke, CVCE, Luxembourg 14
  • 16. Pipelining the CUbRIK components: Human input from experts Capable of complex tasks:  In-depth knowledge of key actors  Context knowledge allows inferences But: Different motivational models!  Public goods  Reputation Lars Wieneke, CVCE, Luxembourg 15
  • 17. Usage for historians  No one truth in history but interpretation, context and discussion  Therefore need to represent ambivalence, contradictions and discussion  Close ties between data representation (Social graph) and their original context (primary sources) Lars Wieneke, CVCE, Luxembourg 16
  • 18. Conclusion  Challenges  What is truth? Humanities vs. Computer Science  Gathering requirements for tools that haven‘t been developed yet  Engaging crowds  Image copyrights  Scientific value?  Refinement of the application  Additional datasources  Improvement of the interface  Integration of the new components Lars Wieneke, CVCE, Luxembourg 17
  • 21. Outlook histoGraph CUbRIK Presentation 20
  • 22. Visit us on WWW.CUBRIKPROJECT.EU @CUBRIKPROJECT CUbRIK Presentation 21 Or follow us on Twitter

Editor's Notes

  1. Ok, let’s start with a joke to relax you guys a bit: How can you tell that a computer scientist received communications training?
  2. Kern des Problems: keine automatische Funktionalität, daher CUbRIK Ansatz mit Humans in the loop Dann Bullet: open toolbox Integration verschiedener Modelle Auf Grundherausforderung beziehen Nicht nur auf Personen beziehen, allgemeiner auf Events/Orten Auf Grundlage der CUbRIK Entwicklungen… Weiterentwicklung! Neue Ansätze durch Dritte
  3. Kern des Problems: keine automatische Funktionalität, daher CUbRIK Ansatz mit Humans in the loop Dann Bullet: open toolbox Integration verschiedener Modelle Auf Grundherausforderung beziehen Nicht nur auf Personen beziehen, allgemeiner auf Events/Orten Auf Grundlage der CUbRIK Entwicklungen… Weiterentwicklung! Neue Ansätze durch Dritte
  4. Technical challenges: Vielleicht doch nach Bedeutung sortieren! Auf das gesamte Bild beziehen, Ausblick integrieren. Noch zu stark auf Identities ausgerichtet, was kann man damit machen, wenn man diese hat? Visual knowledge discovery/exploration