In this paper, we investigate the uniqueness of the phase retrieval problem for the fractional Fourier transform (FrFT) of variable order. This problem occurs naturally in optics and quantum physics. More precisely, we show that if $u$ and $v$ are such that fractional Fourier transforms of order $\alpha$ have same modulus $|F_\alpha u|=|F_\alpha v|$ for some set $\tau$ of $\alpha$'s, then $v$ is equal to $u$ up to a constant phase factor. The set $\tau$ depends on some extra assumptions either on $u$ or on both $u$ and $v$. Cases considered here are $u$, $v$ of compact support, pulse trains, Hermite functions or linear combinations of translates and dilates of Gaussians. In this last case, the set $\tau$ may even be reduced to a single point ({\it i.e.} one fractional Fourier transform may suffice for uniqueness in the problem).