This paper investigates the nonlinear analysis, in the form of nonlinear modes, of mechanical systems undergoing unilateral frictionless contact. The nonlinear eigenproblem is introduced by a Rayleigh quotient minimization with inequality constraints formulated in the frequency domain. An augmented Lagrangian procedure is used for the calculation of the nonlinear contact forces. The efficiency of the proposed method for large-scale mechanical systems involving non-smooth nonlinear terms is shown. An industrial application consisting of a compressor blade in contact with a rigid casing is proposed. Sensitivity of the nonlinear modal parameters to contact is illustrated.