Compiling a benchmarking test-suite for combinatorial black-box optimization - Archive ouverte HAL Access content directly
Conference Papers Year : 2018

Compiling a benchmarking test-suite for combinatorial black-box optimization

Ofer Shir
  • Function : Author
Carola Doerr
Thomas Bäck
  • Function : Author


This contribution focuses on the challenge of formulating a set of benchmark problems and/or a test-suite for Combinatorial Optimization problems when treated as black-box global optimization problems. We discuss the involved dilemmas and possible obstacles of such a compilation. To this end, we formulate a list of design questions that need to be answered as a first step in this compilation process. We articulate our perspective on these questions by proposing a rough classification of relevant problem classes, answering the posed questions, and suggesting a preliminary set of problems. While this position paper addresses the Evolutionary Computation community, it intends to offer an open-minded Computer Science perspective - by considering the broad definition of Combinatorial Optimization and by accounting for equivalent threads within Operations Research and Mathematical Programming communities. At the same time, this work capitalizes on prior art in algorithms' benchmarking, including the authors' own experience with the continuous BBOB benchmark problem set, as well as a number of discrete black-box optimization challenges frequently encountered in practice.
No file

Dates and versions

hal-01921068 , version 1 (13-11-2018)



Ofer Shir, Carola Doerr, Thomas Bäck. Compiling a benchmarking test-suite for combinatorial black-box optimization. GECCO '18 - Genetic and Evolutionary Computation Conference Companion, Jul 2018, Kyoto, Japan. pp.1753-1760, ⟨10.1145/3205651.3208251⟩. ⟨hal-01921068⟩
112 View
0 Download



Gmail Facebook X LinkedIn More