In this paper, we investigate the notion of incorporating feature computation time measures during feature selection in a boosted cascade people detector utilizing heterogeneous pool of features. We present various approaches based on pareto-front analysis, computation time weighted Adaboost, and Binary Integer Programming (BIP) with comparative evaluations. The novel feature selection method proposed based on BIP – the main contribution – mines heterogeneous features taking both detection performance and computation time explicitly into consideration. The results demonstrate that the detector using this feature selection scheme exhibits low miss rates with significant boost in frame rate. For example, it achieves a 2.6% less miss rate at 10e−4 FPPW compared to Dalal and Triggs HOG detector with a 9.22x speed improvement. The presented extensive experimental results clearly highlight the improvements the proposed framework brings to the table.