Research on the biological determinants of male homosexual preference has long realized that the older brother effect (FBOE, i.e., a higher fraternal birth rank of homosexuals) and the antagonist effect (AE, i.e., more fertile women have a higher chance of having a homosexual son) can both generate family data where homosexual men have more siblings and more older siblings than heterosexual men. Various statistical approaches were proposed in the recent literature to evaluate whether the action of FBOE or AE could be discriminated from empirical data, by controlling for the other effect. Here, we used simulated data to formally compare all the approaches that we could find in the relevant literature for their ability to reject the null hypothesis in the presence of a specified alternative hypothesis (tests based on regression, Bayesian modeling, or contingency tables). When testing for the FBOE, the relative performance of the different tests was different depending on the specific function generating the older brother effect. Even if no tests were found to always perform better than the others, some tests performed systematically poorly, and some tests displayed a systematic high rate of type-I error. For testing the AE, the relative performance of the tests was generally not changed across all parameter values assayed, providing a clear ranking of the various proposed approaches. Pros and cons for each candidate test are discussed, taking into consideration power and the rate of type-I error but also practicability, the possibility to control for confounding variables, and to consider alternative hypotheses.