Hyper-parameter optimization tools comparison for Multiple Object Tracking applications
Résumé
Commonly, when developing an algorithm it is necessary to define a certain number of variables that control its behavior. Optimal parameters result in better performance that could translate into profits for companies, stand out among similar applications or better ranking on algorithm competitions. However, it is not a simple task to find the combination of parameters that provides the best results. Manual tuning could be a stressful and difficult task even for expert users. In this paper we present, evaluate and compare several tools in the literature for hyper-parameter optimization. We focus on 4 tools that have been selected due to their number of citations, code availability and impact on literature: MCMC, SMAC, TPE and Spearmint. We analyze these tools in the context of Multi Object Tracking (MOT) that have not been well studied in the literature. MOT itself has been well-studied topic with multiple parameters to be tuned. We evaluate these tools using public benchmarks such as PETS09 or ETH and using the publicly available source code provided by the authors. We analyze the impact of these tools in terms of stability, performance, and usabil-ity, among others. Our results show how the use of these tools change the performance of the application and how this would affect the results of real ranked competitions. Our goal is (1) to encourage the reader to use these tools and (2) to provide a guide that helps to choose the most pertinent tool.
Origine | Fichiers produits par l'(les) auteur(s) |
---|
Loading...