Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Understanding bias in video news & news filtering algorithms


Published on

a joint demonstrator plan within the #responsible #datascience program @NLeSC @NWO

Published in: Technology
  • Be the first to comment

  • Be the first to like this

Understanding bias in video news & news filtering algorithms

  1. 1. @CaptureBias P1/P2: Shared user group & data collection for understanding bias in video news & news filtering algorithms Lora Aroyo, Alec Badenoch, Alessandro Bozzon, Antoaneta Dimitrova, Markus de Jong, Claudia Hauff, Joris Hoboken, Panagiotis Mavridis, Johan Oomen, Jesse de Vos, Claes de Vreese
  2. 2. @CaptureBias P1 FairNews Team Nieuws voorziening in een Big Data Data tijdperk Claes de Vreese, UvA/CW Claudia Hauff, TU Delft Joris van Hoboken, UvA/IvIR Dimitrios Bountouridis, TU Delft
  3. 3. @CaptureBias P2: Capture Bias Team Lora Aroyo, (coordinator) VU Amsterdam, Computer Science Alessandro Bozzon, TU Delft CS & Delft Data Science Alec Badenoch, Utrecht University, Media & Culture Studies Antoaneta Dimitrova, Leiden University, Institute of Public Administration Markus de Jong, VU Amsterdam Post-doc in CS Panagiotis Mavridis, TU Delft Post-doc in CS & Data Science Johan Oomen, Netherlands Institute for Sound and Vision Jesse de Vos, Netherlands Institute for Sound and Vision
  4. 4. @CaptureBias P1/P2 Shared Goals & Vision - P1 and P2 have shared goals and visions - At the same time focusing on different aspects - P1: fairness - P2: accuracy - Study both bias & social sorting - explore their similarities & differences - with respect to their manifestation in the content and social impact - Both projects - target news - could share data & use groups - We propose a two stage plan to work towards a joint demonstrator and show added value of the synergy
  5. 5. @CaptureBias Phase 1 (year 1) ● work on a joint dataset annotated by crowd and experts ● with target annotations, e.g. entities, events, sentiment, relevance, opinions ● aim for a joint publication of this dataset as a resource paper ○ as a great reference as a benchmark dataset ○ as a basis for a research community challenge related to bias in terms of fairness and accuracy. ● work towards discovery of alignment points between the two use cases, e.g. ○ user information needs, interaction and context of use. ○ these will be further used in the second phase.
  6. 6. @CaptureBias Phase 2 (year 2) ● work on reusing algorithms & code across the two projects ● P1 will develop recommendation algorithms for fairness in information filtering ● P2 will develop metrics for accuracy & ambiguity in bias-aware data ● validation opportunity ○ experiment with these algorithms across the two projects ○ observe empirically the interplay between fairness and accuracy ● P2 introduces also cultural and language diversity in the news sphere ○ experiments both with news video and text. ○ code sharing in Phase 2 & data sharing in Phase 1 ⇒ compare results in the different use cases ○ opportunity for another shared publication. ● P1 and P2 demonstrators with reused or shared features
  7. 7. @CaptureBias Joint Demonstrator P1/P2 ● Joint dataset in Y1 ○ Baselines - establishing bias parameters between viewers with diverse political views or background through an evaluation workshops with citizens and experts ○ Github repository, python notebooks, API ● Joint user testing in Y1/Y2 ○ Bias in algorithms & content ○ Accuracy & Transparency metrics and strategies ○ Diversity awareness ● Joint publications (e.g. benchmarks, resource) ○ Y1: reqs for transparency & accuracy (e.g. journalists, social sciences, media scholars) ○ Y2: results from user testing ● Joint dissemination (e.g. presentations & joint video) ○ Media Cafe (Hilversum), SPUI25, Amsterdam Data Science Meetup, Delft DS ○ Dutch Journalism Fund and CLICKNL Media & ICT events ○ Dutch ministries and NGOs events with experts from the policy analysis field
  8. 8. @CaptureBias Synergy ● Joint Dataset ○ annotated by crowds ○ validated by experts ○ Including both text & video news ● Joint Interactive Demo / Video ○ Bias-aware Data Analysis of the content ○ Bias analysis of recommender algorithms ○ Linking bias and fairness ○ Experimenting with transparency ● Partnership with related initiatives ○ at national level ○ at international level