In this paper, we propose a novel machine learning algorithm called quality of service upper confidence bound (QoS-UCB) for the opportunistic spectrum access (OSA) scenario. The proposed algorithm selects an optimal channel in terms of occupancy and quality, e.g. signal to noise ratio (SNR) for transmission. It allows secondary users (SU) to learn the spectrum not only on the vacancy point of view but also on the expected transmission quality by selecting two distinguishable exploration coefficients. Our contribution is threefold: i) We propose a new learning algorithm achieving optimal trade-off between exploration and exploitation when the OSA scenario is modeled as a rested Markov multi-armed bandit (MAB) problem. ii) We state that under mild conditions on the state transition probabilities of Markov chains, the regret of the QoS-UCB policy behaves logarithmically over time. iii) We numerically compare our scheme with an existing UCB1 in OSA context and also show that the QoS-UCB outperforms traditional UCB1 in terms regret.