Using molecular dynamics simulation of a standard coarse-grained polymer glass model we investigate by means of the stress-fluctuation formalism the shear modulus $\mu$ as a function of temperature $T$ and sampling time $\Delta t$. While the ensemble-averaged modulus $\mu(T)$ is found to decrease continuously for all $\Delta t$ sampled, its standard deviation $\delta \mu(T)$ is non-monotonous with a striking peak at the glass transition. Confirming the effective time-translational invariance of our systems, $\mu(\Delta t)$ can be understood using a weighted integral over the shear-stress relaxation modulus $G(t)$. While the crossover of $\mu(T)$ gets sharper with increasing $\Delta t$, the peak of $\delta \mu(T)$ becomes more singular. % It is thus elusive to predict the modulus of a single configuration at the glass transition.