Uncertainties in model projections of future climate change are high, and have not decreased substantially over the last 30 years: IPCC AR4 range of climate sensitivity - the increase in globally averaged surface temperature expected for a doubling of CO2 - is [1.5 °C, 4.5 °C]. The causes and nature of this uncertainty, more specifically whether or not it will be possible to reduce it, is subject to debate. Before digging into this problem, we start by focusing on a basic, somewhat epistemological question: what is uncertainty in the context of climate science, and how should it be quantified? While this question may at first appear trivial, we claim that it has occasionally proved to be a source of confusion in the aforementioned debate. We illustrate this point by revisiting one among the most significant contributions to this debate in the recent past (Roe and Baker, 2007) and highlight the fact that its conclusions are dramatically affected by the chosen definition of uncertainty. To resolve this issue, we propose widely used standard deviation as a unique, broadly applicable definition of uncertainty. We then recall a general result on the propagation of uncertainty available from Probability theory under this definition, and analyse its implication on a simple stochastic Climate toy model. This analysis suggests that high uncertainty is not an inevitable and general consequence of the nature of the climate system, and hence is not doomed to remain high. More specifically, it suggests that reducing uncertainty on feedbacks in GCMs, through an improved understanding of involved physical processes, does lead to a reduction of uncertainty on climate sensitivity projected by models. Finally, we elaborate on foreseeable advances in Climate research and modeling that may lead to a decrease of feedback uncertainty, and subsequently to a reduction of uncertainty on climate sensitivity in the near future.