Swimming consists by definition in propelling through a fluid by means of bodily movements. Thus, from a mathematical point of view, swimming turns into a control problem for which the controls are the deformations of the swimmer. The aim of this paper is to present a unified geometric approach for the optimization of the body deformations of so-called driftless swimmers. The class of driftless swimmers includes, among other, swimmers in a 3D Stokes flow (case of micro-swimmers in viscous fluids) or swimmers in a 2D or 3D potential flow. A general framework is introduced, allowing the complete analysis of five usual nonlinear optimization problems to be carried out. The results are illustrated with examples coming from the literature and with an in-depth study of a swimmer in a 2D potential flow. Numerical tests are also provided.