The paper explores in detail the use of dynamic adaptation gain/learning rate (DAG) for improving the performance of adaptive feedforward attenuation schemes. The DAG is an ARMA (poles-zeros) filter embedded in the gradient type adaptation/learning algorithms and generalizes the various improved gradient algorithms available in the literature. After introducing the DAG algorithm in the context of adaptive feedforward attenuation schemes and providing relationship with other algorithms, its design is developed. Strictly Positive Real (SPR) conditions play an important role in the design of the DAG. Then the stability issues for adaptive/learning systems using a DAG are discussed. The potential of the DAG is then illustrated by experimental results obtained on a relevant adaptive active noise control system.