Most research in data mining and knowledge discovery relies heavily on the availability of datasets. With the rapid growth of user generated content on the internet, there is now an abundance of sources from which data can be drawn. Compared to the amount of work in the field on techniques for pattern discovery and knowledge extraction, there has been relatively little effort directed at the study of effective methods for collecting and evaluating the quality of data.

Human computation is a new research area that studies the process of channeling the vast internet population to perform tasks or provide data towards solving difficult problems that no known efficient computer algorithms can yet solve. There are various genres of human computation applications available today. Games with a purpose (e.g., the ESP Game) specifically target online gamers who, in the process of playing an enjoyable game, generate useful data (e.g., image tags). Crowdsourcing marketplaces (e.g. Amazon Mechanical Turk) are human computation applications that coordinate workers to perform tasks in exchange for monetary rewards. In identity verification tasks, users need to perform some computation in order to access some online content; a recent example of such a human computation application is reCAPTCHA, which leverages millions of users who solve CAPTCHAs every day to correct words in books that optical character recognition (OCR) programs fail to recognize with certainty.

The goal of HCOMP 2009 is to bring together academic and industry researchers in a stimulating discussion of existing human computation applications and future directions of this new subject area. We solicit papers related to various aspects of both general human computation techniques and specific applications, e.g. general design principles; implementation; cost-benefit analysis; theoretical approaches; privacy and security concerns; and incorporation of machine learning / artificial intelligence techniques. An integral part of this workshop will be a demo session where participants can showcase their human computation applications. Specifically, topics of interests include, but are not limited to:

-Abstraction of human computation tasks into taxonomies of mechanisms
-Theories about what makes some human computation tasks fun and addictive
-Differences between collaborative vs. competitive tasks
-Tools and platforms to support human computation
-Domain-specific implementation challenges in human computation games
-Cost versus reliability of labelers
-Benefits of one-time versus repeated labeling
-Game-theoretic mechanism design of incentives for motivation and honest reporting
-Design of manipulation-resistance mechanisms in human computation
-Effectiveness of CAPTCHAs
-Concerns regarding the protection of labeler identities
-Active learning from imperfect human labelers
-Creation of intelligent bots in human computation games
-Limitations of human computation


Preliminary Workshop Format

-presentations by authors
-talks by invited speakers
-poster / demo session


Submission Information

-Papers and extended abstracts should be prepared as PDF files using the KDD conference-paper format and submitted via the CMT system.
-Long papers should be at most nine pages; short papers at most four pages; demo submissions should include either a previously published paper or a one page extended abstract about the demo.


Important Dates

April 18, 2009 Paper submission due
May 16, 2009 Acceptance announcement
May 22, 2009 Camera-ready paper submission due
June 28, 2009 Half-day Workshop at KDD


Program Committee

Serge Belongie (University of California at San Diego)
Laura Dabbish (Carnegie Mellon University)
Ralf Herbrich (Microsoft Research, Cambridge, UK)
John Langford (Yahoo! Research)
David Parkes (Harvard University)
Zoran Popovic (University of Washington)
Paul Resnick (University of Michigan)
Victor Sheng (New York University)
Alexander Sorokin (University of Illinois at Urbana-Champaign)


Organizing Committee

Paul Bennett (Microsoft Research)
Raman Chandrasekar (Microsoft Research)
Max Chickering (Microsoft Corporation)
Panos Ipeirotis (New York University)
Edith Law (Carnegie Mellon University)
Anton Mityagin (Microsoft Corporation)
Foster Provost (New York University)
Luis von Ahn (Carnegie Mellon University)

Contact email: hcomp2009@gmail.com