posted by system || 2795 views || tracked by 10 users: [display]

ICML ML Evaluation 2009 : ICML 2009 Workshop on Evaluation Methods for Machine Learning IV


When Jun 18, 2009 - Jun 18, 2009
Where Montreal, Canada
Submission Deadline Apr 16, 2009
Notification Due Apr 30, 2009
Categories    machine learning

Call For Papers

Call For Submissions to ICML 2009 Workshop
The Fourth Workshop on Evaluation Methods for Machine Learning
June 18, 2009 Montreal Canada.

William Klement, Chris Drummond, Nathalie Japkowicz, and Sofus Macskassy.

The fourth in a series, this workshop intends to continue the debate within the
machine learning community into how we evaluate new algorithms. We aim to discuss
what properties of an algorithm need to be evaluated (e.g., accuracy,
comprehensibility, conciseness); to solicit views and suggestions for other
approaches than those currently used; to investigate alternate methods that could
be useful.

In the course of three previous workshops, the debate has evolved to focus around
the following issues which have captured the interest of the community:

* the role of experiments in evaluation

* the use of one, community wide, evaluation measure (e.g., Accuracy, AUC,

* the relevance of statistical tests to evaluation

* the effectiveness of the UCI data sets for evaluation

* the need for sharing and characterizing benchmark data sets in general

* how to promote the views of this workshop to the rest of the community

The 2008 ICML workshop concluded with agreement that we, as a scientific community,
should substantially change how evaluation is performed in machine learning. We,
however, disagreed on the direction that this change should take. As a continuation
of the same theme, we aim to solicit views, intuitions and visions of alternatives
to change existing evaluation methods. We hope to make progress but still carry
forward the good methods and experiences we already have acquired.

We invite position and technical papers concerning the following issues:

* advantages of existing evaluation methods

* critiques of current evaluation practices

* intuitive and creative alternatives

* performance evaluation issues of concern to the community

* future directions and evolutions of evaluation methods in machine

This list certainly does not capture all the issues worthy of discussion nor the
possible positions. We expect, and very much encourage, position papers raising
other issues that members of the machine learning community think are important.

The day-long workshop consists of:

Invited talks: we plan to have several invited speakers. Some, from
outside of our research community, will be able to criticize our accepted
practices from an external point of view. Some, from inside our community,
will discuss how we could improve on our current practices.

Panel Discussion: our invited speakers will be asked to engage each other on
the various issues surrounding the problem of evaluation in Machine Learning
at the end of the workshop. The audience will be strongly encouraged to
participate in the discussion.

Presentations: The papers accepted to the workshop will be presented throughout
the day between the various invited talks. Papers will be grouped by theme, in
order to facilitate discussion at the end of each session.

Workshop attendance is open to the public and is estimated at 20-25 attendees.
Priority will be given to those active participants in the workshop (paper
authors or speakers).

Authors are invited to submit papers on the issues listed above or other
related positions. Technical or position reports will be considered for this
workshop. To promote a lively event, with plenty of discussion, the organizers
are very interested in papers taking strong positions on the above topics.
Workshop papers should be at least 1 page long and should not exceed 4 pages using
the ICML'09 Style. Submissions should be made electronically in PDF or Postscript
format and should be sent (no later than April 8, 2009 -- EXTENDED to April 16, 2009) by email to:
William Klement, email:
klement -at- site -dot- uottawa -dot- ca

Each submissions will be reviewed by two members of the organizing committee.
A notification of acceptance will be sent by: April 30, 2009. Authors will be
asked to submit a final version of their submission to be posted on the workshop

Important Dates
April 8, 2009 (Extended to April 16, 2008) Submission due date
April 30, 2009 Author notification Date
June 1, 2009 Final version ready
June 14-18, 2009 ICML 2009 Conference
June 18, 2009 Workshop Date

William Klement (main contact)
SITE, University of Ottawa
800 King Edward Ave.
P.O. Box 450, Stn A
Ottawa, ON K1N 6N5, Canada.
Telephone: (613) 562-5800 ext. 6699
Fax: (613) 562-5664

Chris Drummond
NRC Institute for Information Technology, Canada
Email: chris.drummond -at- nrc-cnrc -dot- gc -dot- ca

Nathalie Japkowicz
School of Information Technology and Engineering
University of Ottawa, Canada
Email: nat -at- site -dot- uottawa -dot- ca

Sofus A. Macskassy
Fetch Technologies
United States of America
Email: sofmac -at- fetch -dot- com

Workshop URL

Related Resources

ICML 2020   37th International Conference on Machine Learning
ICDM 2021   21th Industrial Conference on Data Mining
ICML 2021   International Conference on Machine Learning
MLDM 2021   17th International Conference on Machine Learning and Data Mining
ICML WS: Optimization for ML 2020   ICML 2020 Workshop: Beyond First Order Methods in Machine Learning
IARCE 2021-Ei Compendex & Scopus 2021   2021 5th International Conference on Industrial Automation, Robotics and Control Engineering (IARCE 2021)
LREC 2020   12th Conference on Language Resources and Evaluation
CFDSP 2021   2021 International Conference on Frontiers of Digital Signal Processing (CFDSP 2021)
FL-ICML 2020   International Workshop on Federated Learning for User Privacy and Data Confidentiality in Conjunction with ICML 2020
ML_BDA 2021   Special Issue on Machine Learning Technologies for Big Data Analytics