2013.10.22 Tom De Nies - Assessing Content Value for Digital Publishing through Relevance and Provenance-based Trust

606 views

Published on

These are the slides of my talk at the ISWC2013 Doctoral Consortium. http://iswc2013.semanticweb.org/content/doctoral-consortium-program

For more info, go to http://users.ugent.be/~tdenies or follow me on Twitter @TomDeNies

By
Tom De Nies
Supervised by: Erik Mannens and Rik Van de Walle
Ghent University – iMinds – MMLab

Published in: Technology, Business
  • Be the first to comment

  • Be the first to like this

2013.10.22 Tom De Nies - Assessing Content Value for Digital Publishing through Relevance and Provenance-based Trust

  1. 1. ELIS – Multimedia Lab Assessing Content Value for Digital Publishing through Relevance and Provenance-based Trust Tom De Nies Supervised by: Erik Mannens and Rik Van de Walle Ghent University – iMinds – MMLab http://users.ugent.be/~tdenies @TomDeNies (this presentation has links, click them to find out more!) Presented at ISWC 2013 Doctoral Consortium
  2. 2. ELIS – Multimedia Lab Problem Description Digital publishers are facing an information overload … but they get less time to deal with it! They need a way to select content that is valuable to themselves, and to their target audience! Assessing Content Value for Digital Publishing through Relevance and Provenance-based Trust Tom De Nies 22/10/2013 2
  3. 3. ELIS – Multimedia Lab State of the Art Most work on quality assessment of machine-observed data Only limited number of works[1] for human-generated content Very diverse: worthiness, novelty, readability, pageranking, … No integrated, automated approach to assess content value through multiple aspects. Assessing Content Value for Digital Publishing through Relevance and Provenance-based Trust Tom De Nies 22/10/2013 3
  4. 4. ELIS – Multimedia Lab Research Questions and Hypotheses How can we automatically assess the value of content on the Web? Relevance assessment? Trustworthiness assessment? Through provenance + reputation? Retrieval? Reconstruction? Through context and semantic similarity Assessing Content Value for Digital Publishing through Relevance and Provenance-based Trust Tom De Nies 22/10/2013 4
  5. 5. ELIS – Multimedia Lab Proposed Approach A contextual model is used to generate the content's relevance, reconstruct its provenance, and assess its trustworthiness. Assessing Content Value for Digital Publishing through Relevance and Provenance-based Trust Tom De Nies 22/10/2013 5
  6. 6. ELIS – Multimedia Lab Evaluation Plan Content Value depends on end-user and use case makes a general evaluation infeasible representative use cases! E.g. online news Relevance: benchmarks and established metrics (e.g. MediaEval) Provenance reconstruction: Currently no benchmarks or gold standard datasets! -> used preliminary dataset -> make our own (e.g. with Git2PROV) Trust & Value: human evaluation & crowdsourcing (e.g. Amazon MT) Assessing Content Value for Digital Publishing through Relevance and Provenance-based Trust Tom De Nies 22/10/2013 6
  7. 7. ELIS – Multimedia Lab Preliminary Results - Linking videos based on textual content - Supervised method (2012): MAP 0.17 - Unsupervised method (2013): MAP 0.0375 HOWEVER: MAP is wrong metric! - Precision: 35% of top-10 links relevant [AVALON] - 224 news abstracts - Newsworthiness criteria identified (from expert literature) - Detect these criteria automatically - Detected with 83,9% precision [APROVeD] - 420 news items - Reconstruct their provenance (find the original source) - Using our clustering method - 73% of sources found - With 68.2% precision Assessing Content Value for Digital Publishing through Relevance and Provenance-based Trust Tom De Nies 22/10/2013 7
  8. 8. ELIS – Multimedia Lab Reflections Relevance value + Our strong points: Trust Combination Fine-grained Cross-domain - Our points of attention: Annotate uncertainty! [2] Supervised vs. unsupervised Benchmarks Assessing Content Value for Digital Publishing through Relevance and Provenance-based Trust Tom De Nies 22/10/2013 8

×