We study the valuation of variable annuities for an insurer. We concentrate on two types of these contracts that are the guaranteed minimum death benefits and the guaranteed minimum living benefits ones and that allow the insured to withdraw money from the associated account. As for many insurance contracts, the price of variable annuities consists in a fee, fixed at the beginning of the contract, that is continuously taken from the associated account. We use a utility indifference approach to determine this fee and, in particular, we consider the indifference fee rate in the worst case for the insurer i.e. when the insured makes the withdrawals that minimize the expected utility of the insurer. To compute this indifference fee rate, we link the utility maximization in the worst case for the insurer to a sequence of maximization and minimization problems that can be computed recursively. This allows to provide an optimal investment strategy for the insurer when the insured follows the worst withdrawals strategy and to compute the indifference fee. We finally explain how to approximate these quantities via the previous results and give numerical illustrations of parameter sensibility.