Aerodynamic design of projectiles is crucial to ensure that projectiles have the best performance during their flight. Performing aerodynamic design boils down to determining the optimal values of certain design variables of the projectile as the solution of a nonlinear optimization problem involving the stability derivatives of the projectile. It therefore often requires expensive resources, mainly associated to the experiments or numerical simulations needed to obtain these stability derivatives for different values of the design variables. In this paper, a (cheap to evaluate) neural network surrogate model is used to model the stability derivatives. A procedure balancing exploitation and exploration is then devised to determine, based on that surrogate model, the values of the design variables for which the stability derivatives have to be evaluated to both improve the surrogate model and approach the optimal design of the projectile. This framework is applied to optimize the geometrical configuration of a rectangular finner for a given flight scenario.