Convolutional neural networks (CNN) and Long short-term memory (LSTM) provide stateof-the-art performance in various tasks. However, these models are faced with overfitting on small data and cannot measure uncertainty, which have a negative effect on their generalization abilities. In addition, the prediction task can face many challenges because of the complex long-term fluctuations, especially in time series datasets. Recently, applying Bayesian inference in deep learning to estimate the uncertainty in the model prediction was introduced. This approach can be highly robust to overfitting and allows to estimate uncertainty. In this paper, we propose a novel approach using Bayesian inference in a hybrid CNN-LSTM model called CNN-Bayes LSTM for time series prediction. The experiments have been conducted on two real time series datasets, namely sunspot and weather datasets. The experimental results show that the proposed CNN-Bayes LSTM model is more effective than other forecasting models in terms of Root Mean Square Error (RMSE) and Mean Absolute Error (MAE) as well as for uncertainty quantification.