Crowdsourcing (September/October 2012)

Final submissions due: 20 Jan. 2012
Publication date: Sept./Oct. 2012

Please email the guest editors a brief description of the article you plan to submit by 4 January 2011.



Crowdsourcing research aims to understand and build systems that involve humans as part of a larger computational structure. These systems leverage human intelligence to take on challenges that are difficult for algorithms. Examples include games with a purpose (such as the ESP Game), massive volunteer projects like Wikipedia, crowd control mechanisms, new user study methodologies, and interactive systems powered by crowds.

Over the past few years, we’ve observed a proliferation of related workshops, new courses, and tutorials, scattered across many conferences. The participants in this effort cross many disciplines — machine learning, mechanism and market design, information retrieval, decision-theoretic planning, optimization, and human computer interaction. This issue is an opportunity to bring together articles from many backgrounds to speak to a common theme.

This field, while nascent, might encompass numerous topics, including

building and studying systems powered by human computation and the contributions of crowds;
studying how to use crowdsourcing as a tool in user-facing systems, such as user testing, data coding, formative evaluation, or controlled experiments;
studying crowdsourcing platforms like Mechanical Turk, and engineering better platforms; and
designing incentives and mechanisms to improve output quality.

Questions that we seek to answer and we hope the submissions will address include

Methods: What are techniques for solving harder problems than crowds can currently accomplish? How can computation and artificial intelligence guide crowds to achieve their goals faster or more accurately?
Quality: How can we design mechanisms for querying crowds in a way that encourages truthful responses? How can we aggregate noisy outputs from multiple raters?
Architecture: How should we design the crowdsourcing platforms of the future? Can we assign tasks to people to match their particular expertise and interests, or pursue other goals?
Programming: What are some programming paradigms for designing algorithms that effectively leverage the humans in the loop?
Understanding: How do we build human computation systems that involve the joint efforts of both machines and humans, trading off each of their particular strengths and weaknesses?


Questions?

Contact Guest Editors: Ed H. Chi and Michael Bernstein (ic5-2012@computer.org)

All submissions must be original manuscripts of fewer than 5,000 words, with each figure counting as 250 words, focused on Internet technologies and implementations. All manuscripts are subject to peer review on both technical merit and relevance to IC’s international readership—primarily system and software design engineers. We do not accept white papers, and we discourage strictly theoretical or mathematical papers. To submit a manuscript, please log on to ScholarOne (https://mc.manuscriptcentral.com:443/ic-cs) to create or access an account, which you can use to log on to IC’s Author Center and upload your submission.