*** Apologies for cross-posting. Appreciate if you can distribute the CFP. ***
******************************************************************************
IEEE WCCI 2020 Special Session on
Benchmarking of Evolutionary Algorithms for Discrete Optimization
(BEADO)
19-24th July, 2020, Glasgow (UK)
******************************************************************************

IMPORTANT DATES
Paper Submission Deadline: 15 January 2020
Notification of Acceptance: 15 March 2020
Camera-Ready Copy Due: 15 April 2020
Author Registration: 15 April 2020
Conference Presentation: 19-24 June 2020


1. Introduction

The Special Session on Benchmarking of Evolutionary Algorithms for Discrete Optimization (BEADO), a part of the IEEE Congress on Evolutionary Computation (IEEE CEC) at IEEE World Congress on Computational Intelligence (WCCI) 2020, is cordially inviting the submission of original and unpublished research papers.

Evolutionary Computation (EC) is a huge and expanding field, attracting more and more interests from both academia and industry. It includes a wide and ever-growing variety of optimization algorithms, which, in turn, are applied to an even wider and faster growing range of different problem domains, including discrete optimization. For the discrete domain and application scenarios, we want to pick the best algorithms. Actually, we aim to do more, we want to improve upon the best algorithm. This requires a deep understanding of the problem at hand, the performance of the algorithms we have for that problem, the features that make instances of the problem hard for these algorithms, and the parameter settings for which the algorithms perform the best. Such knowledge can only be obtained empirically, by collecting data from experiments, by analyzing this data statistically, and by mining new information from it. Benchmarking is the engine driving research in the fields of EAs for decades, while its potential has not been fully explored.


2. Scope

The goal of this special session is to solicit original works on the research in benchmarking: Works which contribute to the domain of benchmarking of discrete algorithms from the field of Evolutionary Computation, by adding new theoretical or practical knowledge. Papers which only apply benchmarking are not in the scope of the special session.

This special session wants to bring together experts on benchmarking, evolutionary computation algorithms, and discrete optimization. It provides a common forum for them to exchange findings, to explore new paradigms for performance comparison, and to discuss latest issues.


4. List of main topics

Modelling of algorithm behaviors and performance.
Visualizations of algorithm behaviors and performance.
Statistics for performance comparison (robust statistics, PCA, ANOVA, statistical tests, ROC).
Evaluation of real-world goals such as algorithm robustness, and reliability.
Theoretical results for algorithm performance comparison.
Comparison of theoretical and empirical results.
New benchmark problems.
The comparison of algorithms in non-traditional scenarios such as:
multi- or many-objective domains;
parallel implementations, e.g., using GGPUs, MPI, CUDA, clusters, or running in clouds;
large-scale problems or problems where objective function evaluations are costly;
dynamic problems or where the objective functions involve randomized simulations or noise.
Comparative surveys with new ideas on:
dos and don'ts, i.e., best and worst practices, for algorithm performance comparison;
tools for experiment execution, result collection, and algorithm comparison; and
benchmark sets for certain problem domains and their mutual advantages and weaknesses.


5. Names of the organizers of the special session

Ales Zamuda (ales.zamuda@um.si)
Tome Eftimov (tome.eftimov@ijs.si)
Stjepan Picek (stjepan@computer.org), (s.picek@tudelft.nl)


6. Short biography of all organizers

Ales Zamuda is an Assistant Professor and Researcher at University of Maribor (UM), Slovenia. He received Ph.D. (2012), M.Sc. (2008), and B.Sc. (2006) degrees in computer science from UM. His roles also include management committee (MC) membership for Slovenia at European Cooperation in Science (COST), actions CA15140 (ImAppNIO - Improving Applicability of Nature-Inspired Optimisation by Joining Theory and Practice) and IC1406 (cHiPSet - High-Performance Modelling and Simulation for Big Data Applications). He is IEEE Senior Member, IEEE CIS Slovenia Section Chair, IEEE Young Professionals Chair for Slovenia Section, ACM SIGEVO member, ImAppNIO Benchmarks working group vice-chair, and editorial board member (associate editor) for Swarm and Evolutionary Computation (2018 IF=6.330). His areas of computer science applications include evolutionary algorithms, multicriterion optimization, artificial life, ecosystems, and computer animation; currently yielding h-index 18, 52 publications, and 1107 citations on Scopus. He won IEEE R8 SPC 2007 award, IEEE CEC 2009 ECiDUE, 2016 Danubius Young Scientist Award, and 1% top reviewer at 2017 Publons Peer Review Awards, including reviews for research projects, 45 journals, and 80 conferences. He was general conference chair of SEMCCO & FANCCO, and co-organizer of BEADO alikes at GECCO, PPSN, and CEC.

Tome Eftimov is a researcher at Jozef Stefan Institute, Ljubljana, Slovenia. He worked as a postdoctoral researcher at Stanford University, where in parallel was also a research associate at University of California, San Francisco. His main areas of research include statistics, heuristic optimization, natural language processing, knowledge representation, and machine learning. He was awarded his PhD degree from the Jozef Stefan International Postgraduate School, Ljubljana, Slovenia, in 2018. He is involved in courses on probability and statistics, and statistical data analysis. His work related to Deep Statistical Comparison for benchmarking stochastic optimization algorithms was presented as tutorial (i.e. IJCCI 2018, IEEE SSCI 2019) or invited lecture to several international conferences and universities. In 2019, he was awarded as the best young scientist from the president of North Macedonia. In 2018 and 2019, his work related to benchmarking was selected as Hot-Off-the-Press track at GECCO.

Stjepan Picek is assistant professor in the Cyber Security research group of the Faculty of Electrical Engineering, Mathematics and Computer Science at Delft University of Technology, The Netherlands. In July 2015, he completed his PhD at Radboud University Nijmegen, The Netherlands and Faculty of Electrical Engineering and Computing, Zagreb, Croatia. After that, he first worked as a postdoctoral researcher at KU Leuven, Belgium and after that, at CSAIL/MIT, USA. Stjepan also worked for a number of years in industry. His main research interests are at the intersection of cryptography, evolutionary computation, and machine learning. He is IEEE Senior Member and IEEE CIS Croatia Section Chair.