Low-rank matrix completion aims to recover matrices with missing entries and has attracted considerable attention from machine learning researchers. Most of the existing methods, such as weighted nuclear-norm-minimization-based methods and QR-decomposition-based methods, cannot provide both convergence accuracy and convergence speed. To investigate a fast and accurate completion method, an iterative QR-decomposition-based method is proposed for computing an approximate Singular Value Decomposition (CSVD-QR). This method can compute the largest r(r > 0) singular values of a matrix by iterative QR decomposition. Then, under the framework of matrix tri-factorization, a CSVD-QR-based L2,1-norm minimization method (LNM-QR) is proposed for fast matrix completion. Theoretical analysis shows that this QR-decomposition-based method can obtain the same optimal solution as a nuclear norm minimization method, i.e., the L2,1-norm of a submatrix can converge to its nuclear norm. Consequently, an LNM-QR-based iteratively reweighted L2,1-norm minimization method (IRLNM-QR) is proposed to improve the accuracy of LNM-QR. Theoretical analysis shows that IRLNM-QR is as accurate as an iteratively reweighted nuclear norm minimization method, which is much more accurate than the traditional QR-decomposition-based matrix completion methods. Experimental results obtained on both synthetic and real-world visual datasets show that our methods are much faster and more accurate than the state-of-the-art methods.