Acceleration of ultrathin plasma foils by laser radiation pressure promises compact alternatives to the conventional ion accelerators. It was shown, that a major showstopper for such schemes is a strong transverse instability, which develops the surface ripples, and is often attributed to the Rayleigh-Taylor (RT) type. However, simulations indicate, that these perturbations develop the features, that cannot be consistently explained by the RT mechanism. Here we develop a three-dimensional (3D) theory of this instability, which shows that its linear stage is mainly driven by strong electron-ion coupling, while the RT contribution is actually weak. Our model provides the instability spectral structure and its growth rate, that agrees with the large scale 3D particle-in-cell simulations. Numerical modeling shows, that target destruction results from a rapid plasma heating induced by the instability field. Possible paths to instability mitigation are discussed.