Current-induced spin-orbit torques provide a versatile tool for switching magnetic devices. In perpendicular magnets, the dampinglike component of the torque is the main driver of magnetization reversal. The degree to which the fieldlike torque assists the switching is a matter of debate. Here we study the switching of magnetic tunnel junctions with a Co-Fe-B free layer and either W or Ta underlayers, which have a ratio of fieldlike to dampinglike torque of 0.3 and 1, respectively. We show that the fieldlike torque can either assist or hinder the switching of Co-Fe-B when the static in-plane magnetic field required to define the polarity of spin-orbit torque switching has a component transverse to the current. In particular, the noncollinear alignment of the field and current can be exploited to increase the switching efficiency and reliability compared with the standard collinear alignment. By probing individual switching events in real time, we also show that the combination of transverse magnetic field and fieldlike torque can accelerate or decelerate the reversal onset. We validate our observations using micromagnetic simulations and extrapolate the results to materials with different torque ratios. Finally, we propose device geometries that leverage the fieldlike torque for density increase in memory applications and synaptic weight generation.