In this paper, we consider the problem of identifying, by means of boundary element methods and nonlinear optimization, a cavity or obstacle of unknown location and shape embedded in a linearly acoustic or elastic medium. The unknown shape is classically sought so as to achieve a best fit between the measured and computed values of some physical quantity, which is here the scattered acoustic pressure field. One is usually led to the minimization of a cost function J . Classical nonlinear optimization algorithms need the repeated computation of the gradient of the cost function with respect to the design variables as well as the cost function itself. The present paper emphasizes the formulation and effectiveness of the adjoint problem method for the gradient evaluation. First, the hard obstacle inverse problem for 3-D acoustics is considered. For a given J , the adjoint problem is established, and the gradient of J is then formulated in terms of both primary and adjoint states. Next, the adjoint variable approach is extended to the case of a penetrable obstacle in a 3-D acoustical medium, and also for a traction-free cavity in a 3-D elastic medium. Explicit formulae for the gradient of J with respect to shape variations, which appear to be rather compact and elegant, are established for each case. The formulation is incorporated in an unconstrained minimization algorithm, in order to solve numerically the inverse problem. Numerical results are presented for the search of a rigid bounded obstacle embedded in an infinite 3-D acoustic medium, where the measurements are taken to be values of the pressure field on a remote measurement surface, the obstacle being illuminated by monochromatic plane waves. They demonstrate the efficiency of the proposed method. Some computational issues (accuracy, CPU time, influence of measurements errors) are discussed. Finally, for the sake of completeness, the direct differentiation approach is also treated and new derivative BIE formulations are established.