The term Brunet-Derrida behavior refers to the 1997 paper by E. Brunet and B. Derrida "Shift in the velocity of a front due to a cutoff" (see the reference within the paper), where it is shown, based on numerical simulations and heuristic arguments, that a certain branching-selection particle system on the line exhibits the following behavior: as $N$ goes to infinity, the asymptotic velocity of the system with $N$ particles converges to a limiting value at the surprisingly slow rate $(\log N)^{-2}$. In this paper, we consider a class of branching-selection particle systems on $\R$ with $N$ particles, defined through iterated branching-selection steps of the following type. During a branching step, each particle is replaced by two new particles, whose positions are shifted from that of the original particle by independently performing two random walk steps, according to some distribution $p$. During the selection step that follows, only the $N$ rightmost particles are kept among the $2N$ particles obtained at the branching step, to form a new population of $N$ particles. Under generic assumptions on $p$, it is shown that Brunet-Derrida behavior holds for the corresponding particle system. The proofs are based on ideas and results by R. Pemantle, and by N. Gantert, Y. Hu and Z. Shi, and rely on a comparison of the particle system with a family of $N$ independent branching random walks killed below a linear space-time barrier. The results presented here both improve and generalize upon previous work by the first author of this paper, which was completed just before the results by Gantert, Hu and Shi became publicly available.