In the most general case, source localization has to take into account the radiation pattern of the sources of interest. This is particularly important when the sensors surround the sources, and the sources are anisotropic, as is the case in several applications (EEG, speech, musical instruments, etc.). Cramér-Rao bounds for the joint estimation of the position of a source and its radiation pattern are computed for simple cases of sensor array geometries and source models, showing that a good match between the source and the model improves the Cramér-Rao bounds. It is also shown that, in general, using a model more complex than the source makes the Fisher information matrix singular. These results are supported by numerical simulations and physical interpretations.