We study the design of portfolios under a minimum risk criterion. The performance of the optimized portfolio relies on the accuracy to the estimated covariance matrix of portfolio asset returns. For large portfolios, the sample size is often of similar order to the number of assets, and the traditional sample covariance matrix performs poorly. Additionally, financial market data often involve outliers and exhibit heavy-tails, which, if not correctly handled, may further corrupt the covariance estimation. We aim to address these problems by studying the performance of a hybrid covariance matrix estimator based on Tyler's robust M-estimator and on Ledoit-Wolf's shrinkage estimator. Employing recent results from random matrix theory, we develop a consistent estimator of a scaled version of the portfolio risk, based on which, the shrinkage intensity is directly optimized to minimize the risk. Our portfolio optimization method is shown via simulations to outperform existing methods both for synthetic data and for a real market data set from Hang Seng Index.