PartiallySeparableNLPModels.jl: A Julia framework for partitioned quasi-Newton optimization
Résumé
In optimization applications, objectives and constraints often have a partially-separable structure: they are a sum of so-called element functions that depend on a small number of variables.
It is well known that twice-differentiable functions with a sparse Hessian are partially-separable.
We present a framework based on the JuliaSmoothOptimizers ecosystem to model and detect partially-separable problems, and to efficiently evaluate their derivatives by informing AD engines of their structure.
We also implement partitioned trust-region quasi-Newton methods in which Hessian approximations are also informed by the partially-separable structure.
Such methods were proposed in the 1980s and have rivaled limited-memory quasi-Newton methods in efficiency since.
Our implementations feature novel aspects, including the automatic detection of convexity and its impact on the choice of a quasi-Newton approximation, and the use of limited-memory approximations for element functions that depend on a sufficiently large number of variables, which consumes significantly less memory than the dense matrices used in the 1980s and 1990s.
Our numerical results illustrate the performance of a number of variants on a standard problem collection and demonstrate the viability of our methods for problems in which element functions depend on both a small or a substantial number of variables.