This paper proposes an uncertainty quantification framework that enables the analyst to compute statistically calibrated confidence bounds of the reliability of coherent systems, even in the case in which the data are limited. More specifically, we propose to use confidence boxes to do so. Such a proposal is motivated by the fact that confidence boxes offer a guarantee on statistical performances regardless of the amount of available data through repeated use, which is especially useful when considering the fact that reliability or failure data for the reliability analysis are often limited in availability. The aim of this work is to provide tools that allow the analyst to obtain the true confidence intervals over the system failure and reliability at any desired level. This paper first reviews the basics of confidence boxes and reliability analysis before providing general computation tools. From this, a mathematical formalism is presented that relates the component configurations with the corresponding Boolean logic expressions to perform a forward propagation of the confidence boxes under varying dependencies between the components. The feasibility of the proposed framework is then demonstrated through three case studies involving complex systems under varying engineering settings in the form of the (1) pressurized tank system, (2) Training, Research, Isotopes, General Atomics (TRIGA) nuclear research reactor cooling system, and (3) bridge structure system. Through case studies, the validity of our studies is empirically shown, and an evaluation of the strengths and limitations of the proposed framework is presented. Finally, this paper provides perspectives on the future research works that can be undertaken. To provide a better understanding of the proposed framework, open R source code to reproduce the results and perform other related studies is available on GitHub.