Pearlmutter, Barak A. and Siskind, Jeffrey Mark (2008) Reverse-Mode AD in a Functional Framework: Lambda the Ultimate Backpropagator. ACM Transactions on Programming Languages and Systems, 30 (2). 7.1-7.36. ISSN 0164-0925
PDF
BP_Reverse-Mode.pdf
Download (354kB)
BP_Reverse-Mode.pdf
Download (354kB)
Abstract
We show that reverse-mode AD (Automatic Differentiation)—a generalized gradient-calculation
operator—can be incorporated as a first-class function in an augmented lambda calculus, and
therefore into a functional-programming language. Closure is achieved, in that the new operator
can be applied to any expression in the augmented language, yielding an expression in that
language. This requires the resolution of two major technical issues: (a) how to transform nested
lambda expressions, including those with free-variable references, and (b) how to support self
application of the AD machinery. AD transformations preserve certain complexity properties,
among them that the reverse phase of the reverse-mode AD transformation of a function have
the same temporal complexity as the original untransformed function. First-class unrestricted
AD operators increase the expressive power available to the numeric programmer, and may have
significant practical implications for the construction of numeric software that is robust, modular,
concise, correct, and efficient.
Item Type: | Article |
---|---|
Additional Information: | The work of B. A. Pearlmutter was supported, in part, by Science Foundation Ireland grant 00/PI.1/C067 and the Higher Education Authority of Ireland. The work of J. M. Siskind was supported, in part, by National Science Foundation (NSF) grant CCF-0438806. © ACM, 2008. This is the author's version of the work. It is posted here by permission of ACM for your personal use. Not for redistribution. The definitive version was published in ACM Transactions on Programming Languages and Systems (TOPLAS) Volume 30 Issue 2, March 2008, http://doi.acm.org/10.1145/1330017.1330018 |
Keywords: | closures; derivatives; forward-mode AD; higher-order AD; higher-order functional languages; Jacobian; program transformation; reflection; |
Academic Unit: | Faculty of Science and Engineering > Computer Science Faculty of Science and Engineering > Research Institutes > Hamilton Institute |
Item ID: | 3504 |
Depositing User: | Barak Pearlmutter |
Date Deposited: | 29 Feb 2012 14:25 |
Journal or Publication Title: | ACM Transactions on Programming Languages and Systems |
Publisher: | Association for Computing Machinery (ACM) |
Refereed: | Yes |
Funders: | Higher Education Authority of Ireland, National Science Foundation (NSF), Science Foundation Ireland |
Related URLs: | |
URI: | https://mu.eprints-hosting.org/id/eprint/3504 |
Use Licence: | This item is available under a Creative Commons Attribution Non Commercial Share Alike Licence (CC BY-NC-SA). Details of this licence are available here |
Repository Staff Only (login required)
Downloads
Downloads per month over past year