While the usage of automatic differentiation in ML is ubiquitous,
automatic sparse differentiation (ASD) remains largely unknown.
This post introduces ASD, explaining how sparsity can be leveraged to vastly accelerate the computation of Jacobians and Hessians.
Work with Guillaume Dalle and Alexis Montoison at ICLR 2025.