Improved Hessian approximation with modified quasi-Cauchy relation for a gradient-type method

Abstract

In this work we develop a new gradient-type method with improved Hessian approximation for unconstrained optimization problems. The new method resembles the Barzilai-Borwein (BB) method, except that the Hessian matrix is approximated by a diagonal matrix rather than the multiple of the identity matrix in the BB method. Then the diagonal Hessian approximation is derived based on the quasi-Cauchy relation. To further improve the Hessian approximation, we modify the quasi-Cauchy relation to carry some additional information from the values and gradients of the objective function. Numerical experiments show that the proposed method yields desirable improvement.