Analysis of combustion instabilities relies in most cases on linear analysis but most observations of these processes are carried out in the nonlinear regime where the system oscillates at a limit cycle. The objective of this paper is to deal with these two manifestations of combustion instabilities in a unified framework. The flame is recognized as the main nonlinear element in the system and its response to perturbations is characterized in terms of generalized transfer functions which assume that the gain and phase depend on the amplitude level of the input. This ‘describing function’ framework implies that the fundamental frequency is predominant and that the higher harmonics generated in the nonlinear element are weak because the higher frequencies are filtered out by the other components of the system. Based on this idea, a methodology is proposed to investigate the nonlinear stability of burners by associating the flame describing function with a frequency-domain analysis of the burner acoustics. These elements yield a nonlinear dispersion relation which can be solved, yielding growth rates and eigenfrequencies, which depend on the amplitude level of perturbations impinging on the flame. This method is used to investigate the regimes of oscillation of a well-controlled experiment. The system includes a resonant upstream manifold formed by a duct having a continuously adjustable length and a combustion region comprising a large number of flames stabilized on a multipoint injection system. The growth rates and eigenfrequencies are determined for a wide range of duct lengths. For certain values of this parameter we find a positive growth rate for vanishingly small amplitude levels, indicating that the system is linearly unstable. The growth rate then changes as the amplitude is increased and eventually vanishes for a finite amplitude, indicating the existence of a limit cycle. For other values of the length, the growth rate is initially negative, becomes positive for a finite amplitude and drops to zero for a higher value. This indicates that the system is linearly stable but nonlinearly unstable. Using calculated growth rates it is possible to predict amplitudes of oscillation when the system operates on a limit cycle. Mode switching and instability triggering may also be anticipated by comparing the growth rate curves. Theoretical results are found to be in excellent agreement with measurements, indicating that the flame describing function (FDF) methodology constitutes a suitable framework for nonlinear instability analysis.