The document describes a qualitative content analysis project being conducted using Amazon Mechanical Turk. Workers on Mechanical Turk will be presented with questionnaires to classify reviews based on whether they contain information or emotional support. The results from the workers will then be analyzed using qualitative content analysis techniques to identify patterns in the reviews. Python and R will be used to generate models and analyze the data collected from Mechanical Turk workers.

1. Qualitative Content Analysis
Ricky Bilakhia
Advisor – Prof. Aron Lindberg
Approach
Content analysis (CA) is a technique for systematically describing written, spoken or visual
communication and provides quantitative (numerical) description.
This type of analysis has gained popularity because other automated techniques prove to be
expensive or are unavailable in other cases. And is used to spot patterns or peculiar qualities in the
given data that automatic analysis can easily miss.
Future Work
Also we will be using Python for classification because it provides a number of packages which
automates the analysis and has a high accuracy rate
Business Intelligence & Analytics
http://www.stevens.edu/howe/academics/graduate/business-intelligence-analytics
Executive Summary
“Marketplace for work that requires Human Intelligence”
The emergence of online labor markets makes it far easier to use individual human raters to
evaluate materials for data collection and analysis in the real world.
Mechanical Turk is a well developed, commonly used and has the most information
available to assess its suitability for research.
Mechanical Turk is being used for research in disciplines such as natural language
processing, machine learning and human computer interaction.
Technology
R for generating the statistical model.
Amazon Web Service for the verification of work done through R.
Excel or HTML to fetch inputs and write results of the model.
Current Work
Successfully able to implement the creation, assignment and acquiring the HIT
response from Turkers.
Currently working on analyzing the reactions from the Turkers using “Qualitative
Content analysis”
Content Analysis has a wide appeal as providing information about the subjective
dimension of texts. It can be regarded as a classification technique, either binary
(polarity classification into positive/negative) or multi-class categorization (e.g.
positive/neutral/negative).
Purpose
•In a quest to develop a solution which will perform content analysis on a given source, we
came up with a system which is explained in the figure below.
•This solution is a holistic approach to extract, load, host, conduct, refine and analyze number
of user reviews based on Qualitative Content Analysis principle.
•The system is being developed in two different parts where the first part deals with extracting
user comments from any specified source and convert it to a .csv file which will form input to
our system.
•Our system will be responsible for reading these reviews from the csv file and create
questionnaires to present to the turkers.
•These questionnaires are then turned into HITs for our workers with help of Mechanical
Turk.
•Mechanical Turk also will help us filter the genuine workers from the spammers.
•Once we have all the results from the workers, its time to move to the analysis of reviews at
hand.
Research Stimulus
Data about a Research Stimulus:
This possibility involves that the researcher is
studying some collection of objects that need
humans to provide data about them.
 We refer these objects generically as research stimuli;
an individual Turker may work on many stimuli.
In this case, we assume that the data of interest
are inherent in the stimuli (i.e., different individuals
are not expected to have different interpretations) and
require observation with minimal interpretation.
This case describes many uses of AMT for research. For example we will provide
reviews on specific project; the Turkers identify whether they offered information or
emotional support.
Data Source
Deconstruct into
Tasks
Design HITs for Task
Register HITType
Create HIT(s)
Qualification
Requirements
Assignments
Review
Qualify Contact Bonus
Analyze Data
Installation of
MturkR
package and
Its Library.
Requestor
Login to the
AWS through
Credential()
Function
Data Source
for the HIT
Create Single
HIT
Create Bulk HIT
using Template
Submit HIT
Worker Logs
to AWS
Pre-Qualification
Requirement for the
worker
Qualification
Test
Fail
Request to take HIT
Pass
Completes
the HIT
Incomplete
HIT
End
HIT Approval?
HIT Status
Payment
made to
Worker
Accept
End
Rejected
Payment
Received
End
Request HIT
Accepted
End
Rejected
Worker
Contact Worker
Creation of HIT
Accept HIT
Requester
Worker