A presentation by UVA Data Science Institute MSDS 2019 students Charu Rawat, Arnab Sarkar, and Sameer Singh, advised by DSI professor Raf Alvarado and researcher Lane Rasberry, at the 2019 Tom Tom Applied Machine Learning Conference in Charlottesville, VA.
Sequential and reinforcement learning for demand side management by Margaux B...
Wikimedia Foundation, Trust & Safety: Cyber Harassment Classification and Prediction of User Blocks in Wikipedia
1. UVA DSI Capstone Project
Wikimedia Foundation, Trust & Safety
Cyber Harassment Classification and Prediction of User Blocks in Wikipedia
Students - Charu Rawat , Arnab Sarkar, Sameer Singh
Faculty Advisor – Rafael Alvarado
University of Virginia Master’s in Data Science Program
Clients – Patrick Earley , Sydney Poore
Advisor – Lane Rasberry
Trust and Safety Team, Wikimedia Foundation
1
2. The English Wikipedia over the last few years…
• Popularity: Alexa Rank
#5
• Total Edits: 868,717,569
• All Pages: 46,612,179
• Articles 5,767,464
• Registered Users:
35,185,308
• Administrators: ~1,193
https://commons.wikimedia.org/wiki/File:Wikidata_Items_Map_2014_%E2%80%93_2017_gif.gif
2
3. 41% have experienced
personal attacks
66% observed attacks
directed towards others
47% reported a decrease in
contribution and engagement levels
3
Online Harassment at Wikipedia
Pew Research Centre Survey (America centric)
4. Human Driven
Process
Failure to identify
behavior
Bias to overlook
behavior
Inability to
understand context
and scope of
Our goal was to use machine learning to develop a model that can detect abusive content as well as predict
problematic users in the community.
To that end, we leveraged a variety of data to analyse the prevalence and nature of online harassment at scale.
There is a need for a more robust process to combat
harassment, one that can scale well as theWikipedia
community continues to grow in its size and diversity.
Problem
5. • ~1.02 million unique
users have been blocked
Wikipedia till Oct 2018
• 91.5% registered users
• 8.5% are anonymous
users
• Increase in user bocks in
2017, 2018
5
Block User Trends in Wikipedia
7. User Comment Text
Data Repository
on AWS
User Blocks
Identified blocked
users
SQL Queries
Extracted user
comments
Python web scraper
ORES score data
User Activity
Extracted activity
statistics
SQL Queries
Wiki automated
edit quality scores
Python web scraper
Google Ex: Machina
Human annotated wiki
user comments
CSV files
Data Pipeline
8. Model Building Methodology
• Corpus Aggregation + Annotations
• Feature Extraction
Orthographical Features
Char/Word n-gram
NLTK Sentiment Analyzer
Topic Modelling
GloVe Word Embedding
FastText
• Implementing Machine Learning Algorithms
XGBoost – AUC 84% (Best)
• Model Tweaking
8
10. User Risk Model - Early Detection of Problematic Users
FEATURES
Machine
Learning
XGBoost
11. Wikipedia is
full of
morons. I’m
done with
this!
Thanks! This
is helpful
I agree with
him on the
editing trend
F%$@ I will
delete
everything
you post!!!
I don’t think
this is
relevant
How is this
going chicken
shit
0.5
0.3
0.8
0.6
0.1
0.0
WEEKLY STATS
edit count: 5
avg length: 500
active days : 4
WEEKLY STATS
edit count: 7
avg length: 1230
active days : 2
WEEKLY STATS
edit count: 9
avg length: 30
active days : 10
WEEKLY STATS
edit count: 4
avg length: 740
active days : 7
WEEKLY STATS
edit count: 3
avg length: 100
active days : 4
WEEKLY STATS
edit count: 30
avg length: 510
active days : 14
11
Users who indulge in problematic behavior blocked in Wikipedia. About 1.2 mil users have been blocked from 2004 to 2018 , infact the number of blocked users
reached an all time high in 2018. And only ~9% of those blocked users are anonymous contradicting a widespread assumption that anonymity is a major factor when it comes to users doing such things.
Wikipedia vs wikimedia
Fellow students
If we look at the reasons that lead to users getting blocked or the kind of toxic behavior they indulge in that leads to them getting blocked – we can see that
there are some dominant ones like sock puppetry and spam, These reasons have been changing through time,
Also goes on to show that harassment or toxic behavior can take multiple forms
To uncover such insights and build upon them, we analyzed a variety of data made publicly available by Wikipedia. Ranging from structured data pertaining to information about blocked users and general user activity to big data textual in nature relating to user conversations on the platform.
As you can see we used many methods to access the these data sources for our analysis and model building.
