Over the last few years we have observed the emergence of hybrid human-machine information systems which are able to both scale over large amount of data as well as to maintain high-quality data processing intrinsic in human intelligence.
In this talk I will focus on the use of human intelligence at scale by means of crowdsourcing to deal with Big Data problems. We will look specifically on how to deal with the variety in data by means of Human Computation still being able to operate with a large data volume.
First, I will introduce the area of micro-task crowdsourcing also providing an overview of different research challenges that needs to be tackled to enable large-scale hybrid human-machine information systems. Next, I will provide examples of such hybrid systems for entity linking and disambiguation using crowdsourcing and a graph of linked entities as background corpus. I will describe how keyword query understanding can be crowdsourced to build search engines that can answer rare complex queries. Finally, I will present new techniques that allow to improve the quality of crowdsourced information system components by means of push crowdsourcing.