Dempster-Shafer evidence theory, also called the theory of belief function, is widely used for uncertainty modeling and reasoning. However, when the size and number of focal elements are large, the evidence combination will bring a high computational complexity. To address this issue, various methods have been proposed including the implementation of more efficient combination rules and the simplifications or approximations of Basic Belief Assignments (BBAs). In this paper, a novel principle for approximating a BBA into a simpler one is proposed, which is based on the degree of non-redundancy for focal elements. More non-redundant focal elements are kept in the approximation while more redundant focal elements in the original BBA are removed first. Three types of degree of non-redundancy are defined based on three different definitions of focal element distance, respectively. Two different implementations of this principle for BBA approximations are proposed including a batch and an iterative type. Examples, experiments, comparisons and related analyses are provided to validate proposed approximation approaches.