CUbRIK Project                       Vincenzo Croce           Engineering Ingegneria Informatica S.p.a.“In the same way St...
CUbRIK at a glance   36 month large-scale integrating    project started in October 2011   partially funded by the Europ...
The CUbRIK Project Concept: theCube Metaphor    project concept is based a on cube metaphor         progress proceeds al...
Objective: Advance thearchitecture of multimedia search    Multimedia search: coordinated result of three     main proces...
Objective: Advance the Humanplatform    The CUbRIK vision is to incorporate human capacities     and social interaction ...
What achieved so far    Scenario definition         History of Europe – entities         Fashion Application    Requir...
Objective: Advance thearchitecture of multimedia search         Content processing, query processing and feedback process...
Objective: Advance thearchitecture of multimedia search   Example of CUbRIK Pipeline   conflict - situation during the  ...
CUbRIK architecture 26/9/2012   CUbRIK – International Workkshop Search Computing   10
Objective: Humans in the loop    Problem: the uncertainty of analysis algorithms leads to low     confidence results and ...
Human activities 26/9/2012   CUbRIK – International Workkshop Search Computing   12
CrowdSearch Framework                                      Types of tasks                                      Types of ta...
GWAP FrameworkObjective• extract the core aspects of a mediarefinement exploiting a videogame.      • task gamification - ...
Human-powered trademark logodetection demo   Problems in automatic logo detection :        Object recognition is affecte...
Human-powered trademark logodetection demo    Goal: integrate human and automatic     computation to increase precision a...
Human-powered trademark logodetection demo 26/9/2012   CUbRIK – International Workkshop Search Computing   17
Human-powered entities creations   Problems in automatic entities :        Entity creation can be affected by wrong     ...
Human-powered entities creations   Solution: Word Games        Popular        Fun        Educational use        Playe...
Human-powered entities creations   Crossword generation       select a layout       select a slot       pick a word   ...
Vincenzo Croce                          Vincenzo.croce@eng.it            Thanks for your attentionwww.cubrikproject.eu26/9...
Upcoming SlideShare
Loading in...5
×

2012.09.26.CUbRIK at CHORUS + (the progress)

242
-1

Published on

VIncenzo Croce presents CUbRIK achievements and progress along the research roadmap

Published in: Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
242
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
3
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide
  • .
  • 2012.09.26.CUbRIK at CHORUS + (the progress)

    1. 1. CUbRIK Project Vincenzo Croce Engineering Ingegneria Informatica S.p.a.“In the same way Stanley Kubrick was one of the most innovative and pattern-breaking Directors in the Cinema industry, so the CUbRIK project willintroduce real innovative patterns inside the Multimedia search domain” CUbRIK – International Workkshop Search Computing
    2. 2. CUbRIK at a glance 36 month large-scale integrating project started in October 2011 partially funded by the European Commission’s 7th Framework ICT Programme for Research and Technological Developmentwww.cubrikproject.eu 26/9/2012 CUbRIK – International Workkshop Search Computing 2
    3. 3. The CUbRIK Project Concept: theCube Metaphor  project concept is based a on cube metaphor  progress proceeds along three dimensions: technological, human and business ones  At the same time, the cube metaphor recalls the open composability of the Rubik-cube puzzle 26/9/2012 CUbRIK – International Workkshop Search Computing 3
    4. 4. Objective: Advance thearchitecture of multimedia search  Multimedia search: coordinated result of three main processes:  Content processing: acquisition, analysis, indexing and knowledge extraction from multimedia content  Query processing: derivation of an information need from a user and production of a sensible response  Feedback processing: quality feedback on the appropriateness of search results 26/9/2012 CUbRIK – International Workkshop Search Computing 4
    5. 5. Objective: Advance the Humanplatform  The CUbRIK vision is to incorporate human capacities and social interaction  CUbRIK aims at harvesting different forms of human computation:  Implicit, such as emotion recognition  Individual, such as intelligence and decision making in gaming  Social, such as in crowdsearch 26/9/2012 CUbRIK – International Workkshop Search Computing 5
    6. 6. What achieved so far  Scenario definition  History of Europe – entities  Fashion Application  Requirements  Architecture  H-Demos  CUbRIK Pipelines  CrowdSearch Frameworks  Q&A  GWAP 26/9/2012 CUbRIK – International Workkshop Search Computing 7
    7. 7. Objective: Advance thearchitecture of multimedia search  Content processing, query processing and feedback processing phases implemented by means of independent components  Components are organized in CUbRIK pipelines  Each application defines ad-hoc pipelines that provide unique multimedia search capabilities in that scenario 26/9/2012 CUbRIK – International Workkshop Search Computing 8
    8. 8. Objective: Advance thearchitecture of multimedia search Example of CUbRIK Pipeline conflict - situation during the analysis where absence or contradictions of facts recorded in Annotations may arise Conflict Resolutoin Task  -> Application – GWAP  -> application-performer pair – Q&A application 26/9/2012 CUbRIK – International Workkshop Search Computing 9
    9. 9. CUbRIK architecture 26/9/2012 CUbRIK – International Workkshop Search Computing 10
    10. 10. Objective: Humans in the loop  Problem: the uncertainty of analysis algorithms leads to low confidence results and conflicting opinions on automatically extracted features  Solution: humans have superior capacity for understanding the content of audiovisual material  State of the art: humans replace automatic feature extraction processes (human annotations)  Our contribution: integration of human judgment and algorithms  Goal: improve the performance of multimedia content processing 26/9/2012 CUbRIK – International Workkshop Search Computing 11
    11. 11. Human activities 26/9/2012 CUbRIK – International Workkshop Search Computing 12
    12. 12. CrowdSearch Framework Types of tasks Types of tasks •Automatic tasks •Automatic tasks •Crowd tasks: tasks that are executed by •Crowd tasks: tasks that are executed by an open-ended community of performers an open-ended community of performers 26/9/2012 CUbRIK – International Workkshop Search Computing 13
    13. 13. GWAP FrameworkObjective• extract the core aspects of a mediarefinement exploiting a videogame. • task gamification - converting a task into a game • task injection - integrates the task goals inside pre existing games. Challenges • Drive user’s motivation • Select the proper users • based on their profile, competence and reliability. • Validate the results provided by the crowd 26/9/2012 CUbRIK – International Workkshop Search Computing 14
    14. 14. Human-powered trademark logodetection demo Problems in automatic logo detection :  Object recognition is affected by the quality of the input set of images  Uncertain matches, i.e., the ones with low matching score, could not contain the searched logo 26/9/2012 CUbRIK – International Workkshop Search Computing 15
    15. 15. Human-powered trademark logodetection demo  Goal: integrate human and automatic computation to increase precision and recall w.r.t. fully automatic solutions 26/9/2012 CUbRIK – International Workkshop Search Computing 16
    16. 16. Human-powered trademark logodetection demo 26/9/2012 CUbRIK – International Workkshop Search Computing 17
    17. 17. Human-powered entities creations Problems in automatic entities :  Entity creation can be affected by wrong  synset or gloss  relation between synset and concept  relation between concepts  entity names  entity attribute  attribute value 26/9/2012 CUbRIK – International Workkshop Search Computing 18
    18. 18. Human-powered entities creations Solution: Word Games  Popular  Fun  Educational use  Played with:  Words  real world entity names  definitions (glosses) Finalist - crossword:  extremely popular  fun  words, clues, relations (in clues)  maximum mistake coverage  easy personalization  themes or topics 26/9/2012 CUbRIK – International Workkshop Search Computing 19
    19. 19. Human-powered entities creations Crossword generation  select a layout  select a slot  pick a word  check grid consistency  Feedback , check of  Etype  relations  attributes  values 26/9/2012 CUbRIK – International Workkshop Search Computing 20
    20. 20. Vincenzo Croce Vincenzo.croce@eng.it Thanks for your attentionwww.cubrikproject.eu26/9/2012 CUbRIK – International Workkshop Search Computing 21
    1. A particular slide catching your eye?

      Clipping is a handy way to collect important slides you want to go back to later.

    ×