

Hybrid approaches to adjoint code generation with dco/c++
Ph.D. thesis
  

Author(s)
Johannes Lotz

Year 2016 
Abstract This dissertation is concerned with the computation of arbitraryorder derivative projections (tangents and adjoints) of numerical simulation programs written in C++. Thanks to the increasing computational power, simulation nowadays plays a fundamental role within a wide range of applications. On this basis, in the last decades simulation programs became the foundation for optimization problems in a wide variety of domains, e.g. design optimization in computational fluid dynamics. When using derivativebased algorithms to carry out the optimization, first and higherorder sensitivities of the underlying simulation program are required. Since in many cases gradients of scalar objective functions need to be computed, the adjoint method should be used. Adjoint programs can either be written by hand or generated by Algorithmic (or Automatic) Differentiation (ad) tools. ad is a semantic program transformation exploiting the chain rule of differential calculus to automatically generate programs such that sensitivities are computed in addition to the original function values. Writing adjoint programs by hand can be a tedious, timeconsuming, and errorprone task, but on the other hand, the wellversed programmer can possibly write much more efficient code, especially when exploiting mathematical properties by symbolic transformations. When handcoding an adjoint simulation program, an incremental software development process is difficult. To achieve an effective development process and get an efficient program, hybridization techniques can be used to couple an ad tool with handwritten code. This makes a topdown adjoint code development possible  beginning with a pure ad solution, going step by step to a more efficient implementation.This thesis consists of three main points. First, the software dco/c++ is presented. dco/c++ implements ad by overloading in C++ and features hybridization techniques in a flexible and efficient way. Flexibility is meant in terms of compliance with a modern C++ environment (e.g. generic concepts, exceptionsafety, threadsafety, or portability), and in terms of a convenient interface with emphasis on the support for arbitraryorder derivatives. Efficiency on the other hand is ensured by the newly developed ad test suite, which carries out performance tests also in comparison to other ad tools.As the second main point, arbitraryorder derivative projections of common numerical algorithms like linear and nonlinear solvers are derived symbolically, exploiting mathematical properties. Those results are shown to be of practical relevance by connecting them to the hybridization capabilities of dco/c++. This combination is presented with the help of an extensive case study demonstrating the effectiveness of the approach. In addition, dco/c++ was successfully used in many projects and its applicability is therefore shown in terms of reference projects. When using ad to generate an adjoint program, the resulting executable typically requires a huge amount of memory. The third point of this thesis therefore covers the Call Tree Reversal (CTR) problem. CTR is a checkpointing technique, which allows for a tradeoff between additional runtime and memory requirements. The problem itself is an NPcomplete optimization problem, which in this thesis is approached by a mixed integer programming formulation for the first time. 
AD Tools dco/c++ 
AD Theory and Techniques Checkpointing, Hierarchical Approach, Integration of Analytic Derivatives 
BibTeX
@PHDTHESIS{
Lotz2016Hat,
type = "Dissertation",
url = "http://publications.rwthaachen.de/record/667318",
author = "Lotz, Johannes",
title = "Hybrid approaches to adjoint code generation with dco/c++",
school = "Department of Computer Science, RWTH Aachen University",
year = "2016",
abstract = "This dissertation is concerned with the computation of arbitraryorder derivative
projections (tangents and adjoints) of numerical simulation programs written in C++. Thanks to the
increasing computational power, simulation nowadays plays a fundamental role within a wide range of
applications. On this basis, in the last decades simulation programs became the foundation for
optimization problems in a wide variety of domains, e.g. design optimization in computational fluid
dynamics. When using derivativebased algorithms to carry out the optimization, first and
higherorder sensitivities of the underlying simulation program are required. Since in many cases
gradients of scalar objective functions need to be computed, the adjoint method should be used.
Adjoint programs can either be written by hand or generated by Algorithmic (or Automatic)
Differentiation (AD) tools. AD is a semantic program transformation exploiting the chain rule of
differential calculus to automatically generate programs such that sensitivities are computed in
addition to the original function values. Writing adjoint programs by hand can be a tedious,
timeconsuming, and errorprone task, but on the other hand, the wellversed programmer can possibly
write much more efficient code, especially when exploiting mathematical properties by symbolic
transformations. When handcoding an adjoint simulation program, an incremental software development
process is difficult. To achieve an effective development process and get an efficient program,
hybridization techniques can be used to couple an AD tool with handwritten code. This makes a
topdown adjoint code development possible  beginning with a pure AD solution, going step by step
to a more efficient implementation.This thesis consists of three main points. First, the software
dco/c++ is presented. dco/c++ implements AD by overloading in C++ and features hybridization
techniques in a flexible and efficient way. Flexibility is meant in terms of compliance with a
modern C++ environment (e.g. generic concepts, exceptionsafety, threadsafety, or portability), and
in terms of a convenient interface with emphasis on the support for arbitraryorder derivatives.
Efficiency on the other hand is ensured by the newly developed AD test suite, which carries out
performance tests also in comparison to other AD tools.As the second main point, arbitraryorder
derivative projections of common numerical algorithms like linear and nonlinear solvers are derived
symbolically, exploiting mathematical properties. Those results are shown to be of practical
relevance by connecting them to the hybridization capabilities of dco/c++. This combination is
presented with the help of an extensive case study demonstrating the effectiveness of the approach.
In addition, dco/c++ was successfully used in many projects and its applicability is therefore shown
in terms of reference projects. When using AD to generate an adjoint program, the resulting
executable typically requires a huge amount of memory. The third point of this thesis therefore
covers the Call Tree Reversal (CTR) problem. CTR is a checkpointing technique, which allows for a
tradeoff between additional runtime and memory requirements. The problem itself is an NPcomplete
optimization problem, which in this thesis is approached by a mixed integer programming formulation
for the first time.",
ad_tools = "dco/c++",
ad_theotech = "Checkpointing, Hierarchical Approach, Integration of Analytic Derivatives"
}
 
back

