There exist three main approaches to reduction associated to canonical Lie group actions on a symplectic manifold, namely, foliation reduction, introduced by Cartan, Marsden-Weinstein reduction, and optimal reduction, introduced by the authors. When the action is free, proper, and admits a momentum map these three approaches coincide. The goal of this paper is to study the general case of a symplectic action that does not admit a momentum map and one needs to use its natural generalization, a cylinder valued momentum map introduced by Condevaux, Dazord, and Molino. In this case it will be shown that the three reduced spaces mentioned above do not coincide, in general. More specifically, the Marsden-Weinstein reduced spaces are not symplectic but Poisson and their symplectic leaves are given by the optimal reduced spaces. Foliation reduction produces a symplectic reduced space whose Poisson quotient by a certain Lie group associated to the group of symmetries of the problem equals the Marsden-Weinstein reduced space. We illustrate these constructions with concrete examples, special emphasis being given to the reduction of a magnetic cotangent bundle of a Lie group in the situation when the magnetic term ensures the non-existence of the momentum map for the lifted action. The precise relation of the cylinder valued momentum map with group valued momentum maps for Abelian Lie groups is also given.