This paper proposes an improved semantic based cache model: our method boils down to using the first pass of the automatic speech recognition (ASR) system, associated to con dence scores and semantic fields, for driving the second pass. We use a Driven Decoding Algorithm (DDA), which allows us to combine ASR systems, by guiding the search algorithm of a primary system with an auxiliary system. We propose a strategy that uses DDA to drive a semantic cache, according to the con dence measures. The method works like an unsupervised language model adaptation. Results show, on 8 hours, that semantic-DDA yields signi cant improvements to the baseline: we obtain a 4% word error rate relative improvement without acoustic adaptation, and 1.9% after adaptation.