We investigate the stability of a Newtonian ferrofluid cylinder under a steady magnetic field. Linear stability analysis is performed by imposing a small-amplitude axisymmetric disturbance to the basic state. Admissible magnetic field shapes are found and dispersion relations are obtained for two general shapes: axial and non-axial. For the latter, a wire is needed to avoid a singularity at the cylinder centre. Solutions are plotted for two particular shapes previously discussed in the literature for an inviscid ferrofluid: axial and azimuthal. The applicability of the developed theory to cylinder and jet experiments using a solenoid and a wire carrying a current to create the axial and azimuthal magnetic fields, respectively, is studied. For both cases, results show that the magnetic field has a stabilising effect, as already evidenced. In addition, by solving the equation for the cutoff wavenumber, we show that, unlike the azimuthal case, the axial case cannot be stable for all wavenumbers. Taking into account the ferrofluid viscosity better explains the experimental results on ferrofluid cylinders in an azimuthal magnetic field. By choosing a specific admissible magnetic fields shape, the range of wavenumbers in which the cylinder is unstable can be controlled. A radial magnetic field is one example studied here without providing any details on the means to create the field. This latter case reveals the potential of magnetic fields to control the drop size during the ferrofluid jet breakup. These results may be of interest for applications in the printing and medical fields.