High speed data transmission for wireless communication in orthogonal frequency division multiplexing (OFDM) system requires effective channel state information (CSI). CSI should be precisely estimated with low consumption of spectral resources and acceptable computational cost. To realize this goal, an effective compressed sensing (CS) based channel estimation scheme is proposed for sparse channels with large delay spreads, without prior knowledge of channel statistics and noise standard deviation. By fully considering the rank of the measurement matrix, a novel algorithm based on orthogonal matching pursuit (OMP) and least squares (LS) methods with a new threshold is proposed for effective channel estimation. Simulation results show that with fewer number of pilots, the proposed method outperforms the compared existing channel estimation methods in a comprehensive way and approaches the optimal channel estimation performance.