Publication: Fast Reverse-Mode Automatic Differentiation Using Expression Templates in C++
Introduction
Applications
Tools
Research Groups
Workshops
Publications
   List Publications
   Advanced Search
   Info
   Add Publications
My Account
About

Fast Reverse-Mode Automatic Differentiation Using Expression Templates in C++

- Article in a journal -
 

Author(s)
Robin J. Hogan

Published in
ACM Transactions on Mathematical Software

Year
2014

Abstract
Gradient-based optimization problems are encountered in many fields, but the associated task of differentiating large computer algorithms can be formidable. The operator-overloading approach to performing reverse-mode automatic differentiation is the most convenient for the user but current implementations are typically 10-35 times slower than the original algorithm. In this paper a fast new operator-overloading method is presented that uses the expression template programming technique in C++ to provide a compile-time representation of each mathematical expression as a computational graph that can be efficiently traversed in either direction. Benchmarking with four different numerical algorithms shows this approach to be 2.6--9 times faster than current operator-overloading libraries, and 1.3--7.7 times more efficient in memory usage. It is typically less than 4 times the computational cost of the original algorithm, although poorer performance is found for all libraries in the case of simple loops containing no mathematical functions. An implementation is freely available in the Adept C++ software library.

AD Tools
Adept

BibTeX
@ARTICLE{
         Hogan2014FRM,
       author = "Robin J. Hogan",
       title = "Fast Reverse-Mode Automatic Differentiation Using Expression Templates in {C++}",
       journal = "{ACM} Transactions on Mathematical Software",
       pages = "26:1--26:24",
       url = "http://doi.acm.org/10.1145/2560359",
       accepted = "9 December 2013",
       abstract = "Gradient-based optimization problems are encountered in many fields, but the
         associated task of differentiating large computer algorithms can be formidable. The
         operator-overloading approach to performing reverse-mode automatic differentiation is the most
         convenient for the user but current implementations are typically 10-35 times slower than the
         original algorithm. In this paper a fast new operator-overloading method is presented that uses the
         \textit{expression template} programming technique in C++ to provide a compile-time
         representation of each mathematical expression as a computational graph that can be efficiently
         traversed in either direction. Benchmarking with four different numerical algorithms shows this
         approach to be 2.6--9 times faster than current operator-overloading libraries, and 1.3--7.7 times
         more efficient in memory usage. It is typically less than 4 times the computational cost of the
         original algorithm, although poorer performance is found for all libraries in the case of simple
         loops containing no mathematical functions. An implementation is freely available in the
         \textit{Adept} C++ software library.",
       volume = "40",
       number = "4",
       year = "2014",
       month = "jun",
       ad_tools = "Adept"
}


back
  

Contact:
autodiff.org
Username:
Password:
(lost password)