An infinite horizon stochastic optimal control problem with running maximum cost is considered. The value function is characterized as the viscosity solution of a second-order HJB equation with mixed boundary condition. A general numerical scheme is proposed and convergence is established under the assumptions of consistency, monotonicity and stability of the scheme. A convergent semi-Lagrangian scheme is presented in detail.