ADAHESSIAN: An Adaptive Second Order Optimizer for Machine Learning

Easy-to-use AdaHessian optimizer (PyTorch)

Jax implementation of the AdaHessian optimizer

Analyze AdaHessian optimizer on 2D functions.

Survey on performance between Ada-Hessian vs well-known first-order optimizers on MNIST & CIFAR-10 datasets

Discussion of advantages and disadvantages of AdaHessian, a state-of-the-art Second Order Methods over First Order Methods on a Non-Convex Optimization Problem (digits classification on MNIST database using ResNet18). - @ EPFL