In this paper, we address the problem of structure and motion recovery from two views of a scene containing planes, i.e, sets of coplanar points. Most of the existing works do only exploit this constraint in a sub-optimal manner We propose to parameterize the structure of such scenes with planes and points on planes and derive the MLE (Maximum Likelihood Estimator) using a minimal parameterization based on 2D entities. The result is the estimation of camera motion and 3D structure in projective space, that minimizes reprojection error while satisfying the piecewise planarity. We propose a quasi-linear estimator that provides reliable initialization values for plane equations. Experimental results show that the reconstruction is of clearly superior quality compared to traditional methods based only on points, even if the scene is not perfectly piecewise planar.