Human activity recognition (HAR) based on sensor information has become a hot topic of research due to its wide range of applications in health care, fitness, and smart homes. However, the classification of activities with similar sensor signals such as standing and sitting is usually more challenging for the design of efficient activity recognition algorithms. Considering the characteristic of human activities with different granularity, which can provide complementary knowledge for individual granularity activity recognition, we propose a novel approach that combines different-granularity beliefs based on belief functions (BFs) theory for HAR. Specifically, at first, two support vector machines (SVMs) are trained to acquire different-granularity basic belief assignments (BBAs) of human activity, respectively. Then, a prior knowledge library (PKL) is established and the least square method is employed to solve the transformation matrix for mapping coarse-grained BBAs to fine-grained BBAs. Finally, the classic DSm (DSmC) rule is adopted to combine the two sets of fine-grained BBAs and further make decisions. Several experiments are conducted to illustrate the performance of the proposed method using two widely used University of California, Irvine (UCI) datasets: smartphone and mHealth. The experimental results show that the proposed method efficiently improves the classification accuracy with respect to other related methods.