posted by user: grupocole || 4038 views || tracked by 8 users: [display]

AMT 2010 : NAACL-HLT 2010 Workshop on Amazon Mechanical Turk


When Jun 5, 2010 - Jun 6, 2010
Where Los Angeles, California
Abstract Registration Due Jan 31, 2010
Submission Deadline Mar 1, 2010
Notification Due Mar 30, 2010
Final Version Due Apr 12, 2010
Categories    NLP

Call For Papers

NAACL-HLT 2010 Workshop on Amazon Mechanical Turk
June 5 or 6, Los Angeles, CA

Workshop Website:

Submission Deadline: March 1, 2010
Shared Task Proposal Deadline: January 31, 2010

Creating Speech and Text Language Data With Amazon's Mechanical Turk

Amazon's Mechanical Turk (AMT) is an online labor market that allows
you to pay people small sums of money to do "Human Intelligence Tasks"
or HITs. Tasks include labeling images, listening to short pieces of
audio, researching topics on the internet, and scrubbing database

A number of recent papers have evaluated the effectiveness of using
Mechanical Turk to create annotated data for natural language
processing applications. The availability of Mechanical Turk for
creating annotated speech and text language data can fundamentally
change human language technology tasks. Open questions include:
- How can we ensure high quality annotations?
- What tools are available for obtaining complex annotations?
- What types of annotations and evaluations are possible when the cost
is dramatically reduced?

This work will explore uses of AMT in several ways:
- Shared task: What can you do with $100 and AMT? Participants will be
given a budget to spend on AMT and submit papers describing the
results of their experience.
- General papers: These papers will explore general issues with using
Mechanical Turk for language processing research.

Important Dates
Jan 31, 2010 Shared Task Proposal Deadline (for teams wishing to
be considered for funding)
Mar 1, 2010 Paper submission deadline
Mar 30, 2010 Notification of acceptance
Apr 12, 2010 Camera-ready papers due
Jun 5-6, 2010 Workshop at NAACL in Los Angeles, CA (exact date TBA)

Workshop Website

Organizing Committee
Chris Callison-Burch (Johns Hopkins University)
Mark Dredze (Johns Hopkins University)

Program Committee
Jordan Boyd-Graber (UMD)
Michael Bloodgood (HLTCOE)
Bob Carpenter (Alias-i)
James Dennis (J2 Labs)
Maxine Eskenazi (CMU)
Nikesh Garera (Kosmix)
Jim Glass (MIT)
Alex Gruenstein (Google)
Janna Hamaker (Amazon)
Benjamin Lambert (CMU)
Alexandre Klementiev (JHU)
Ian McGraw (MIT)
Scott Novotney (JHU)
Brendan O'Connor (Dolores Labs)
Massimo Poesio (University of Essex)
Joseph Reisinger (UT Austin)
Ted Sandler (Amazon)
Stephanie Seneff (MIT)
Rion Snow (Stanford / Twitter)

We thank Amazon Mechanical Turk for supporting the shared task.

Related Resources

ACL / NAACL-HLT / EMNLP / COLING 2022   ACL 2022 - NAACL 2022 - COLING 2022 - EMNLP 2022 Joint Call for Tutorials
NCOM 2022   8th International Conference on Networks & Communications
*ACL tutorial 2022   ACL/NAACL-HLT/EMNLP/COLING 2022 tutorial proposals
NMCO 2022   8th International Conference on Networks, Mobile Communication
NAACL 2022   2022 Annual Conference of the North American Chapter of the Association for Computational Linguistics
AIFU 2022   8th International Conference on Artificial Intelligence and Applications
IIPhDW 2023   International Interdisciplinary PhD Workshop (IIPhDW)
ECTIJ 2022   Economics, Commerce and Trade Management: An International Journal
TMAE--Ei Compendex, Scopus 2022   2022 4th International Conference on Trends in Mechanical and Aerospace (TMAE 2022)--Ei Compendex, Scopus
NLDB 2022   27th International Conference on Natural Language & Information Systems