The granular column collapse is a simplified version of granular flows such as landslides, avalanches, and other industrial processes mobilized in air or within a fluid. In this configuration, the particles collapse in an accelerating phase, reaching a state of constant spreading velocity until they decelerate and stop. Granular flows commonly involve particles of different sizes, a property termed polydispersity. Understanding the role of polydispersity remains a challenging task that is often analyzed with nearly monodisperse systems and demanding a series of simplifications when coupled with a fluid in a numerical model. Here, we study the effect of particle-size polydispersity in dry and immersed granular columns, using a finite element method-discrete element method model for fluid-particle interactions. We show that the velocity of the column collapse and runout distance decrease with an increase in the level of polydispersity in immersed conditions, and remain nearly independent of the level of polydispersity in dry conditions. Moreover, we find that the runout scales with the spreading front kinetic energy, weighted by the ratio between the particles' density and the density difference between particles and fluid. This scaling helps in identifying the governing processes in polydisperse granular columns, unifying the runout description of both dry and immersed collapses, and indicating that the column initial packing fraction is the governing parameter.