Strong gravitational lensing produces multiple images of a gravitational wave (GW) signal, which can be observed by detectors as time-separated copies of the same event. It has been shown that under favourable circumstances, by combining information from a quadruply lensed GW with electromagnetic observations of lensed galaxies, it is possible to identify the host galaxy of a binary black hole coalescence. Comparing the luminosity distance obtained through electromagnetic means with the effective luminosity distance inferred from the lensed GW signal would then enable us to constrain alternative theories of gravity that allow for modified GW propagation. Here we analyze models including large extra spatial dimensions, a running Planck mass, and a model that captures propagation effects occurring in a variety of alternative theories to general relativity. We consider a plausible population of lenses and binary black holes and use Bayesian inference on simulated GW signals as seen in current detectors at design sensitivity, to arrive at a realistic assessment of the bounds that could be placed. We find that, due to the fact that the sources of lensed events will typically be at much larger redshifts, this method can improve over bounds from GW170817 and its electromagnetic counterpart by a factor of $\sim 5$ to $\mathcal{O}(10^2)$, depending on the alternative gravity model.