Non-uniform temporal weights (TWs) are often reported regarding the perceptual evaluation of dynamic auditory and visual information, such as perceptual judgments of the overall intensity of time-varying stimuli. In particular, primacy effects, i.e., a stronger influence of early compared to later stimulus information on the perceptual decision, have been observed across a large number of studies. Yet, it is not clear whether these non-uniform patterns of TWs result from sensory or attentional processes that coincidentally produce similar time-courses, or whether they reflect the common signature of supra-modal and subject-specific decision-making processes. The present study addresses the hypothesis that TWs in loudness (perceived auditory intensity) and brightness (perceived visual intensity) judgments result from a common supramodal evidence-integration process. In Experiment 1, we compared TWs for loudness and brightness judgments in the same participants, with task difficulty matched individually. The observed average temporal weighting profiles differed substantially between the two modalities. In Experiment 2, we assessed the additional contribution of modality-specific sensory and attentional processes to the observed differences between TWs by measuring intensity resolution at different temporal positions in the auditory and visual stimuli. We observed a significantly different dependence of sensitivity on temporal position in the two modalities, but these sensitivity differences only partially accounted for the temporal weighting differences observed in Experiment 1. The collective findings indicate that the TWs observed for loudness and brightness judgments cannot be attributed to a supramodal evidence-integration process alone. Instead, our results suggest that both sensory and decision-making processes shape patterns of TWs.