We obtain a quenched vector-valued almost sure invariance principle (ASIP) for random expanding on average cocycles. This is achieved by combining the adapted version of Gou\"{e}zel's approach for establishing ASIP and the recent construction of the so-called adapted norms, which in some sense eliminate the non-uniformity of the decay of correlations. For real-valued observables, we also show that the martingale approximation technique is applicable in our setup, and that it yields the ASIP with better error rates. Finally, we present an example showing the necessity of a scaling condition, answering a question of the first and third authors.