Crowd IQ - Aggregating opinions to boost performance

Yoram Bachrach, Thore Graepel, Gjergji Kasneci, Michal Kosinski, Jurgen Van Gael

Research output: Contribution to conferencePaperpeer-review

34 Scopus citations

Abstract

We show how the quality of decisions based on the aggregated opinions of the crowd can be conveniently studied using a sample of individual responses to a standard IQ questionnaire. We aggregated the responses to the IQ questionnaire using simple majority voting and a machine learning approach based on a probabilistic graphical model. The score for the aggregated questionnaire, Crowd IQ, serves as a quality measure of decisions based on aggregating opinions, which also allows quantifying individual and crowd performance on the same scale. We show that Crowd IQ grows quickly with the size of the crowd but saturates, and that for small homogeneous crowds the Crowd IQ significantly exceeds the IQ of even their most intelligent member. We investigate alternative ways of aggregating the responses and the impact of the aggregation method on the resulting Crowd IQ. We also discuss Contextual IQ, a method of quantifying the individual participant's contribution to the Crowd IQ based on the Shapley value from cooperative game theory.

Original languageEnglish
Pages968-975
Number of pages8
StatePublished - 2012
Externally publishedYes
Event11th International Conference on Autonomous Agents and Multiagent Systems 2012: Innovative Applications Track, AAMAS 2012 - Valencia, Spain
Duration: 4 Jun 20128 Jun 2012

Conference

Conference11th International Conference on Autonomous Agents and Multiagent Systems 2012: Innovative Applications Track, AAMAS 2012
Country/TerritorySpain
CityValencia
Period4/06/128/06/12

Keywords

  • Human intelligence
  • Opinion aggregation
  • Shapley value

Fingerprint

Dive into the research topics of 'Crowd IQ - Aggregating opinions to boost performance'. Together they form a unique fingerprint.

Cite this