Announcement of the BioNLP'09 shared task on Event Extraction

http://www-tsujii.is.s.u-tokyo.ac.jp/GENIA/SharedTask/

We are pleased to announce the BioNLP'09 shared task on Event
Extraction, organized in conjunction with the NAACL-HLT'09 BioNLP
workshop, 4-5 June 2009. The goals of the shared task are to provide
common and consistent task definitions, datasets and evaluation for
biomedical information extraction systems based on rich semantics and
a forum for the presentation of varying but focused efforts on their
development.

*IMPORTANT* In order to establish efficient communication channel
between organizers and potential participants, please send
notification of your interest to bionlpsharedtask@gmail.com as soon as
possible. We will maintain a mailing list for procedural
communication. In the notification mail, please write the name,
affiliation and email address of your contact person.


Tasks:

The main concern of the shared task is the extraction of bio-events
from literature, focusing particularly on molecular events involving
proteins and genes. To concentrate efforts on the novel aspects of the
extraction task, a named entity recognition subtask will not be
included in the shared task. Accordingly, a gold standard set of named
entity annotations will be provided as input, and participants will be
required to extract molecular events concerning these entities. The
shared task is designed to address a semantically rich IE problem as a
whole, divided into three subtasks to allow separate evaluation of the
performance of participating systems of different aspects of the
problem:

Task 1. Event detection and characterization
Task 2. Event argument recognition
Task 3. Recognition of negations and speculations

For more details regarding format and examples, please refer to the shared
task homepage: http://www-tsujii.is.s.u-tokyo.ac.jp/GENIA/SharedTask/


Resources for the Shared task:

The organizers will provide human-curated reference material for the
training and evaluation of participating systems. For training, a
dataset based on the publicly available portion of the GENIA corpus
will be provided in a stand-off format. For evaluation, a held-out
part of the same corpus will be provided with the gold annotation
hidden. The goal for participating systems is to recreate the gold
annotation based on the information induced from the training data.


Tools and external resources:

To facilitate the development of systems making use of the wealth of
available NLP tools, we will provide simplified access to tools such
as taggers and syntactic analyzers. In addition to the resources
provided by the task organizers, participants are encouraged to
develop their own and make use of other external tools and resources.


Proceedings:

Each participating team will be sent their score and the gold standard
answers will be made available to all participants within one day
after the deadline for submission of final results. All participants
are then invited to submit a manuscript describing their approach and
results. We encourage in particular manuscripts that present careful
analysis and discussion of the results and explore previously
under-utilized resources in novel ways.

Manuscripts may be submitted either as full papers (8+1 pages) or
short papers (4 pages), following the submission guidelines of the
NAACL main conference. Any participating team may choose to submit a
full paper, and the teams with the best-performing systems in the
shared task subtasks will be especially encouraged to submit full
papers. All papers will be reviewed by the shared task organizers
using a double-blind review process. Note that based on the reviews,
some authors may be asked to recast their submissions as short papers
or poster papers. The page limits and presentation formats are

Full papers : 8 pages + 1 page for references, oral presentation
Short papers : 4 pages, oral presentation
Poster papers : 2 pages, poster presentation

All accepted papers will be included in the shared task section of the
BioNLP workshop proceedings and the authors will be invited to present
their work at the shared task session of the workshop.


Important Dates:

- Dec 15 : Release of training data sample
- Jan 19 : Release of full training data
- Feb 23 : Release of test data
- Mar 2 : Deadline for submission of results
- Mar 3 : Notification of the results of scoring
- Mar 16 : Paper submission deadline
- Apr 6 : Notification of acceptance
- Apr 20 : Camera-ready paper submission deadline
- Jun 5 : BioNLP workshop special shared task sessions


Shared Task Chair:
Jun'ichi Tsujii, University of Tokyo, University of Manchester, and
National Centre for Text Minding

Shared Task Organizers :
Jin-Dong Kim, University of Tokyo, on behalf of GENIA group
Tomoko Ohta, University of Tokyo, on behalf of GENIA group
Sampo Pyysalo, University of Tokyo, on behalf of BioInfer group
Yoshinobu Kano, University of Tokyo, on behalf of U-Compare initiative