Recently, there has been an increased awareness that simplistic adaptation of performance analysis developed for random real-valued signals and parameters to the complex case may be inadequate or may lead to intractable calculations. Unfortunately, many fundamental statistical tools for handling complex-valued parameter estimators are missing or scattered in the open literature. In this paper, we survey some known results and provide a rigorous and unified framework to study the statistical performance of complex-valued parameter estimators with a particular attention paid to properness (i.e., second order circularity), specifically referring to the second-order statistical properties. In particular, some new properties relative to the properness of the estimates, asymptotically minimum variance bound and Whittle formulas are presented. A new look at the role of nuisance parameters is given, proving and illustrating that the noncircular Gaussian distributions do not necessarily improve the Cramer-Rao bound (CRB) with respect to the circular case. Efficiency of subspace-based complex-valued parameter estimators are presented with a special emphasis is put on noisy linear mixture