Solving a linear system of equations is an important underlying part of numerous scientific applications. In this article, we address the issue of non-deterministic and, therefore, non-reproducible solution of linear systems and propose an approach to ensure its reproducibility. Our approach is based on the hierarchical and modular structure of linear algebra algorithms. Consequently, we divide computations into smaller logical blocks – such as a blocked LU factorization, triangular system solve, and matrix-matrix multiplication – and ensure their reproducible results. In this manner, we also split the blocked LU factorization into the unblocked LU and the BLAS-3 routines; the former is built on top of scaling a vector and outer product of two vectors routines from BLAS-1 and-2, accordingly. In this work, our focus is on constructing these building blocks that eventually lead, as we will prove, to the reproducible solution of linear systems.