Factor analysis has proven to be a relevant tool for extracting tissue time-activity curves (TACs) in dynamic PET images, since it allows for an unsupervised analysis of the data. Reliable and interpretable results are possible only if it is considered with respect to suitable noise statistics. However, the noise in reconstructed dynamic PET images is very difficult to characterize, despite the Poissonian nature of the count rates. Rather than explicitly modeling the noise distribution, this paper proposes to study the relevance of several divergence measures to be used within a factor analysis framework. To this end, the β-divergence, widely used in other applicative domains, is considered to design the data-fitting term involved in three different factor models. The performances of the resulting algorithms are evaluated for different values of β, in a range covering Gaussian, Poissonian, and Gamma-distributed noises. The results obtained on two different types of synthetic images and one real image show the interest of applying non-standard values of β to improve the factor analysis.