When applying a neural network to address a new learning problem, rather than training a network from scratch, it is common practice to utilise a network pre-trained on a related dataset and then fine-tune it to the data of the target task. This poses the question: which pre-trained network should be selected? This work investigates this problem in the context of three different dataset relationships: same-source, same-domain, and cross-domain. We utilise Meta-Album, which offers an extensive collection of datasets from various unrelated domains. We first split each of the 30 datasets of Meta-Album into a meta-train dataset and meta-test dataset, then create pre-trained models for each meta-train dataset, and finally compare the performances of the pre-trained models in a fine-tuning context when applied to meta-test tasks. We categorise the performances into the three dataset relationship groups and find that the same-source category performs best in terms of accuracy. Then, using meta-features calculated on the meta-train dataset and meta-test tasks, we train statistical meta-models that are employed to select the best pre-trained model for a given meta-test task. Our best meta-model identifies the best-performing model in ∼ 25% of the cases. It improves upon a baseline that selects the best average model by more than 30%.