PRELIMINARY CALL FOR PAPERS

ECIR 2009 Workshop on
Contextual Information Access, Seeking and Retrieval Evaluation
April 6, 2009, Toulouse, France
http://www.irit.fr/CIRSE/

AIMS

The main purpose of this workshop is to bring together IR researchers
working on or interested in the evaluation of approaches to contextual
information access, seeking and retrieval, and let them to share their
latest research results, to express their opinions on the related issues,
and to promote discussion on the future directions of evaluation.

BACKGROUND

Since the 1990s, the interest in the notion of context in Information
Access, Seeking and Retrieval increased. Many researchers have been
concerning with the use of context in adaptive, interactive, personalized or
collaborative systems, the design of explicit and implicit feedback
techniques, the investigation of relevance, the application of a notion of
context to problems like advertising or mobile search.

Previous workshops and conferences, i.e. IR in Context (IRiX, 2005),
Adaptive IR (AIR, 2006, 2008), Context-based IR (CIR, 2005, 2007) and
Information Interaction in Context (IIiX, 2006, 2008) gathered researchers
exploring theoretical frameworks and applications which have focussed on
contextual IR systems.

An important issue which gave raise to discussion has been Evaluation.

It is commonly accepted that the traditional evaluation methodologies used
in TREC, CLEF, NTCIR and INEX campaigns are not always suitable for
considering the contextual dimensions in the information seeking/access
process. Indeed, laboratory-based or system oriented evaluation is
challenged by the presence of contextual dimensions such as user
interaction, profile or environment which significantly impact on the
relevance judgments or usefulness ratings made by the end user.

Therefore, new research is needed to understand how to overcome the
challenge of user-oriented evaluation and to design novel evaluation
methodologies and criteria for contextual information retrieval evaluation.
Until now, the experiments and results have been often technology or
application and no standard evaluation methodology or standard test
collections emerged from contextual information access, seeking and
retrieval research.

FORMAT

To discuss and compare different approaches to the topic of the workshop, we
will encourage the presentation and participation of both researchers who
are interested in methodological issues of evaluation and contextual
information access, seeking and retrieval, and practitioners who have
deployed and tested systems that make use of user’s context.

To facilitate the presentation of research activities having a different
maturity, the workshop programme will include both long and short papers
covering a range of evaluation methods, techniques and tools for contextual
information access seeking and retrieval.

To give the young researchers an opportunity to present their results and
on-going research, they will be invited to submit short papers and discuss
their contributions in a less formal way. To this end, a short time will be
devoted to the presentation and a longer time will be devoted to the
discussion.

SUBMISSION

Original and unpublished papers are welcome on any aspect including:

• User, system and context modelling for information access seeking and
retrieval evaluation.
• Evaluation of implicit or explicit feedback techniques.
• Evaluation of personal information retrieval systems.
• Social media and networking based search.
• Learning algorithms that use non-traditional relevance judgments.
• Novel or extension of traditional evaluation measures.
• Novel techniques for collecting document relevance.
• Contextual and user simulation algorithms.
• Design of novel test collections.
• Accuracy evaluation of personal profiles built using implicit set-level
responses.
• Merging ranking from collaborative system outputs.
• Application and evaluation of context-based systems for distributed
retrieval, personal search, digital libraries, archives and museums.
• Application and evaluation of context-based access to television
broadcasted recordings, image, video and music collections.

Short papers and long papers will be respectively 2 pages and 4 pages long
using ACM format
(http://www.acm.org/sigs/publications/proceedings-templates). All submitted
papers will be peer-reviewed by the workshop programme committee. At least
one author of each paper must attend the workshop to present the paper. All
accepted papers will be published in the workshop proceedings. For more
information, please see the workshop website http://www.irit.fr/CIRSE/
Please send the papers to the workshop organizers at cirse@irit.fr

IMPORTANT DATES

Paper Submission: January 30, 2009
Notification of Acceptance: February 20, 2009
Camera-Ready papers due: February 27, 2009

ORGANIZERS

Bich-Liên Doan, Supélec, France, Bich-Lien.Doan@supelec.fr
Joemon Jose, University of Glasgow, UK, jj@dcs.gla.ac.uk
Massimo Melucci, University of Padua, Italy, melo@dei.unipd.it
Lynda Tamine-Lechani, IRIT, France, Lynda.Lechani@irit.fr



INVITED SPEAKERS

To be announced

PROGRAM COMMITTEE
Birger Larsen, Royal School of Library and Information Science, Denmark
Emanuele Di Buccio, University of Padua, Italy
Gilles Falquet, University of Geneva, Switzerland
Nicola Ferro, University of Padua, Italy
Martin Halvey, University of Glasgow, United Kingdom
Hideo Joho, University of Glasgow, United Kingdom
Robert Vila, University of Glasgow, United Kingdom
Gareth Jones, Dublin City University, Ireland
Peter Ingwersen, Royal School of Library and Information Science, Denmark
Dianne Kelly, University of North Carolina, USA
Claude de Loupy, University of Paris X, France
Patrick Paroubek, University of Paris XI, France
Maarten de Rijke, University of Amsterdam, The Netherlands
Mathieu Roche, University of Montpellier, France
Ian Ruthven, University of Strathclyde, United Kingdom
Tassos Tombros, Queen Mary, University of London, United Kingdom
Robert Villa, University of Glasgow, United Kingdom