<?xml version='1.0'?>
<html xmlns='http://www.w3.org/1999/xhtml'
xmlns:math='http://www.w3.org/1998/Math/MathML'
>
<head>
<title>The Theory of Reverse Mode</title>
<meta http-equiv='Content-Type' content='text/html' charset='utf-8'/>
<meta name="description" id="description" content="The Theory of Reverse Mode"/>
<meta name="keywords" id="keywords" content=" the theory of reverse mode taylor notation binary operators addition subtraction multiplication division standard math functions "/>
<style type='text/css'>
body { color : black }
body { background-color : white }
A:link { color : blue }
A:visited { color : purple }
A:active { color : purple }
</style>
<script type='text/javascript' language='JavaScript' src='_reversetheory_xml.js'>
</script>
</head>
<body>
<table><tr>
<td>
<a href="http://www.coin-or.org/CppAD/" target="_top"><img border="0" src="_image.gif"/></a>
</td>
<td><a href="erf_forward.xml" target="_top">Prev</a>
</td><td><a href="expreverse.xml" target="_top">Next</a>
</td><td>
<select onchange='choose_across0(this)'>
<option>Index-></option>
<option>contents</option>
<option>reference</option>
<option>index</option>
<option>search</option>
<option>external</option>
</select>
</td>
<td>
<select onchange='choose_up0(this)'>
<option>Up-></option>
<option>CppAD</option>
<option>Appendix</option>
<option>Theory</option>
<option>ReverseTheory</option>
</select>
</td>
<td>
<select onchange='choose_down3(this)'>
<option>CppAD-></option>
<option>Install</option>
<option>Introduction</option>
<option>AD</option>
<option>ADFun</option>
<option>preprocessor</option>
<option>multi_thread</option>
<option>library</option>
<option>ipopt_solve</option>
<option>Example</option>
<option>speed</option>
<option>Appendix</option>
</select>
</td>
<td>
<select onchange='choose_down2(this)'>
<option>Appendix-></option>
<option>Faq</option>
<option>Theory</option>
<option>glossary</option>
<option>Bib</option>
<option>Bugs</option>
<option>WishList</option>
<option>whats_new</option>
<option>deprecated</option>
<option>compare_c</option>
<option>License</option>
</select>
</td>
<td>
<select onchange='choose_down1(this)'>
<option>Theory-></option>
<option>ForwardTheory</option>
<option>ReverseTheory</option>
<option>reverse_identity</option>
</select>
</td>
<td>
<select onchange='choose_down0(this)'>
<option>ReverseTheory-></option>
<option>ExpReverse</option>
<option>LogReverse</option>
<option>SqrtReverse</option>
<option>SinCosReverse</option>
<option>AtanReverse</option>
<option>AsinReverse</option>
<option>AcosReverse</option>
<option>tan_reverse</option>
<option>erf_reverse</option>
</select>
</td>
<td>
<select onchange='choose_current0(this)'>
<option>Headings-></option>
<option>Taylor Notation</option>
<option>Binary Operators</option>
<option>---..Addition</option>
<option>---..Subtraction</option>
<option>---..Multiplication</option>
<option>---..Division</option>
<option>Standard Math Functions</option>
<option>Contents</option>
</select>
</td>
</tr></table><br/>
<center><b><big><big>The Theory of Reverse Mode</big></big></b></center>
<br/>
<b><big><a name="Taylor Notation" id="Taylor Notation">Taylor Notation</a></big></b>
<br/>
In Taylor notation, each variable corresponds to
a function of a single argument which we denote by
<code><i><font color="black"><span style='white-space: nowrap'>t</span></font></i></code>
(see Section 10.2 of
<a href="bib.xml#Evaluating Derivatives" target="_top"><span style='white-space: nowrap'>Evaluating Derivatives</span></a>
).
Here and below
<math xmlns="http://www.w3.org/1998/Math/MathML" display="inline"><mrow>
<mi mathvariant='italic'>X</mi>
<mo stretchy="false">(</mo>
<mi mathvariant='italic'>t</mi>
<mo stretchy="false">)</mo>
</mrow></math>
,
<math xmlns="http://www.w3.org/1998/Math/MathML" display="inline"><mrow>
<mi mathvariant='italic'>Y</mi>
<mo stretchy="false">(</mo>
<mi mathvariant='italic'>t</mi>
<mo stretchy="false">)</mo>
</mrow></math>
, and
<code><i><font color="black"><span style='white-space: nowrap'>Z(t)</span></font></i></code>
are scalar valued functions
and the corresponding <code><i>p</i></code>-th order Taylor coefficients row vectors are
<math xmlns="http://www.w3.org/1998/Math/MathML" display="inline"><mrow>
<mi mathvariant='italic'>x</mi>
</mrow></math>
,
<math xmlns="http://www.w3.org/1998/Math/MathML" display="inline"><mrow>
<mi mathvariant='italic'>y</mi>
</mrow></math>
and
<math xmlns="http://www.w3.org/1998/Math/MathML" display="inline"><mrow>
<mi mathvariant='italic'>z</mi>
</mrow></math>
; i.e.,
<math xmlns="http://www.w3.org/1998/Math/MathML" display="block"><mrow>
<mtable rowalign="center" ><mtr><mtd columnalign="left" >
<mi mathvariant='italic'>X</mi>
<mo stretchy="false">(</mo>
<mi mathvariant='italic'>t</mi>
<mo stretchy="false">)</mo>
</mtd><mtd columnalign="center" >
<mo stretchy="false">=</mo>
</mtd><mtd columnalign="right" >
<msup><mi mathvariant='italic'>x</mi>
<mrow><mo stretchy="false">(</mo>
<mn>0</mn>
<mo stretchy="false">)</mo>
</mrow>
</msup>
<mo stretchy="false">+</mo>
<msup><mi mathvariant='italic'>x</mi>
<mrow><mo stretchy="false">(</mo>
<mn>1</mn>
<mo stretchy="false">)</mo>
</mrow>
</msup>
<mo stretchy="false">*</mo>
<mi mathvariant='italic'>t</mi>
<mo stretchy="false">+</mo>
<mo stretchy="false">⋯</mo>
<mo stretchy="false">+</mo>
<msup><mi mathvariant='italic'>x</mi>
<mrow><mo stretchy="false">(</mo>
<mi mathvariant='italic'>p</mi>
<mo stretchy="false">)</mo>
</mrow>
</msup>
<mo stretchy="false">*</mo>
<msup><mi mathvariant='italic'>t</mi>
<mi mathvariant='italic'>p</mi>
</msup>
<mo stretchy="false">+</mo>
<mi mathvariant='italic'>O</mi>
<mo stretchy="false">(</mo>
<msup><mi mathvariant='italic'>t</mi>
<mrow><mi mathvariant='italic'>p</mi>
<mo stretchy="false">+</mo>
<mn>1</mn>
</mrow>
</msup>
<mo stretchy="false">)</mo>
</mtd></mtr><mtr><mtd columnalign="left" >
<mi mathvariant='italic'>Y</mi>
<mo stretchy="false">(</mo>
<mi mathvariant='italic'>t</mi>
<mo stretchy="false">)</mo>
</mtd><mtd columnalign="center" >
<mo stretchy="false">=</mo>
</mtd><mtd columnalign="right" >
<msup><mi mathvariant='italic'>y</mi>
<mrow><mo stretchy="false">(</mo>
<mn>0</mn>
<mo stretchy="false">)</mo>
</mrow>
</msup>
<mo stretchy="false">+</mo>
<msup><mi mathvariant='italic'>y</mi>
<mrow><mo stretchy="false">(</mo>
<mn>1</mn>
<mo stretchy="false">)</mo>
</mrow>
</msup>
<mo stretchy="false">*</mo>
<mi mathvariant='italic'>t</mi>
<mo stretchy="false">+</mo>
<mo stretchy="false">⋯</mo>
<mo stretchy="false">+</mo>
<msup><mi mathvariant='italic'>y</mi>
<mrow><mo stretchy="false">(</mo>
<mi mathvariant='italic'>p</mi>
<mo stretchy="false">)</mo>
</mrow>
</msup>
<mo stretchy="false">*</mo>
<msup><mi mathvariant='italic'>t</mi>
<mi mathvariant='italic'>p</mi>
</msup>
<mo stretchy="false">+</mo>
<mi mathvariant='italic'>O</mi>
<mo stretchy="false">(</mo>
<msup><mi mathvariant='italic'>t</mi>
<mrow><mi mathvariant='italic'>p</mi>
<mo stretchy="false">+</mo>
<mn>1</mn>
</mrow>
</msup>
<mo stretchy="false">)</mo>
</mtd></mtr><mtr><mtd columnalign="left" >
<mi mathvariant='italic'>Z</mi>
<mo stretchy="false">(</mo>
<mi mathvariant='italic'>t</mi>
<mo stretchy="false">)</mo>
</mtd><mtd columnalign="center" >
<mo stretchy="false">=</mo>
</mtd><mtd columnalign="right" >
<msup><mi mathvariant='italic'>z</mi>
<mrow><mo stretchy="false">(</mo>
<mn>0</mn>
<mo stretchy="false">)</mo>
</mrow>
</msup>
<mo stretchy="false">+</mo>
<msup><mi mathvariant='italic'>z</mi>
<mrow><mo stretchy="false">(</mo>
<mn>1</mn>
<mo stretchy="false">)</mo>
</mrow>
</msup>
<mo stretchy="false">*</mo>
<mi mathvariant='italic'>t</mi>
<mo stretchy="false">+</mo>
<mo stretchy="false">⋯</mo>
<mo stretchy="false">+</mo>
<msup><mi mathvariant='italic'>z</mi>
<mrow><mo stretchy="false">(</mo>
<mi mathvariant='italic'>p</mi>
<mo stretchy="false">)</mo>
</mrow>
</msup>
<mo stretchy="false">*</mo>
<msup><mi mathvariant='italic'>t</mi>
<mi mathvariant='italic'>p</mi>
</msup>
<mo stretchy="false">+</mo>
<mi mathvariant='italic'>O</mi>
<mo stretchy="false">(</mo>
<msup><mi mathvariant='italic'>t</mi>
<mrow><mi mathvariant='italic'>p</mi>
<mo stretchy="false">+</mo>
<mn>1</mn>
</mrow>
</msup>
<mo stretchy="false">)</mo>
</mtd></mtr></mtable>
</mrow></math>
For the purposes of this discussion,
we are given the <code><i>p</i></code>-th order Taylor coefficient row vectors
<math xmlns="http://www.w3.org/1998/Math/MathML" display="inline"><mrow>
<mi mathvariant='italic'>x</mi>
</mrow></math>
,
<math xmlns="http://www.w3.org/1998/Math/MathML" display="inline"><mrow>
<mi mathvariant='italic'>y</mi>
</mrow></math>
, and
<math xmlns="http://www.w3.org/1998/Math/MathML" display="inline"><mrow>
<mi mathvariant='italic'>z</mi>
</mrow></math>
.
In addition, we are given the partial derivatives of a scalar valued function
<math xmlns="http://www.w3.org/1998/Math/MathML" display="block"><mrow>
<mi mathvariant='italic'>G</mi>
<mo stretchy="false">(</mo>
<msup><mi mathvariant='italic'>z</mi>
<mrow><mo stretchy="false">(</mo>
<mi mathvariant='italic'>j</mi>
<mo stretchy="false">)</mo>
</mrow>
</msup>
<mo stretchy="false">,</mo>
<mo stretchy="false">…</mo>
<mo stretchy="false">,</mo>
<msup><mi mathvariant='italic'>z</mi>
<mrow><mo stretchy="false">(</mo>
<mn>0</mn>
<mo stretchy="false">)</mo>
</mrow>
</msup>
<mo stretchy="false">,</mo>
<mi mathvariant='italic'>x</mi>
<mo stretchy="false">,</mo>
<mi mathvariant='italic'>y</mi>
<mo stretchy="false">)</mo>
</mrow></math>
We need to compute the partial derivatives of the scalar valued function
<math xmlns="http://www.w3.org/1998/Math/MathML" display="block"><mrow>
<mi mathvariant='italic'>H</mi>
<mo stretchy="false">(</mo>
<msup><mi mathvariant='italic'>z</mi>
<mrow><mo stretchy="false">(</mo>
<mi mathvariant='italic'>j</mi>
<mn>-1</mn>
<mo stretchy="false">)</mo>
</mrow>
</msup>
<mo stretchy="false">,</mo>
<mo stretchy="false">…</mo>
<mo stretchy="false">,</mo>
<msup><mi mathvariant='italic'>z</mi>
<mrow><mo stretchy="false">(</mo>
<mn>0</mn>
<mo stretchy="false">)</mo>
</mrow>
</msup>
<mo stretchy="false">,</mo>
<mi mathvariant='italic'>x</mi>
<mo stretchy="false">,</mo>
<mi mathvariant='italic'>y</mi>
<mo stretchy="false">)</mo>
<mo stretchy="false">=</mo>
<mi mathvariant='italic'>G</mi>
<mo stretchy="false">(</mo>
<msup><mi mathvariant='italic'>z</mi>
<mrow><mo stretchy="false">(</mo>
<mi mathvariant='italic'>j</mi>
<mo stretchy="false">)</mo>
</mrow>
</msup>
<mo stretchy="false">,</mo>
<msup><mi mathvariant='italic'>z</mi>
<mrow><mo stretchy="false">(</mo>
<mi mathvariant='italic'>j</mi>
<mn>-1</mn>
<mo stretchy="false">)</mo>
</mrow>
</msup>
<mo stretchy="false">,</mo>
<mo stretchy="false">…</mo>
<mo stretchy="false">,</mo>
<msup><mi mathvariant='italic'>z</mi>
<mrow><mo stretchy="false">(</mo>
<mn>0</mn>
<mo stretchy="false">)</mo>
</mrow>
</msup>
<mo stretchy="false">,</mo>
<mi mathvariant='italic'>x</mi>
<mo stretchy="false">,</mo>
<mi mathvariant='italic'>y</mi>
<mo stretchy="false">)</mo>
</mrow></math>
where
<math xmlns="http://www.w3.org/1998/Math/MathML" display="inline"><mrow>
<msup><mi mathvariant='italic'>z</mi>
<mrow><mo stretchy="false">(</mo>
<mi mathvariant='italic'>j</mi>
<mo stretchy="false">)</mo>
</mrow>
</msup>
</mrow></math>
is expressed as a function of the
<code><i>j-1</i></code>-th order Taylor coefficient row
vector for
<math xmlns="http://www.w3.org/1998/Math/MathML" display="inline"><mrow>
<mi mathvariant='italic'>Z</mi>
</mrow></math>
and the vectors
<math xmlns="http://www.w3.org/1998/Math/MathML" display="inline"><mrow>
<mi mathvariant='italic'>x</mi>
</mrow></math>
,
<math xmlns="http://www.w3.org/1998/Math/MathML" display="inline"><mrow>
<mi mathvariant='italic'>y</mi>
</mrow></math>
; i.e.,
<math xmlns="http://www.w3.org/1998/Math/MathML" display="inline"><mrow>
<msup><mi mathvariant='italic'>z</mi>
<mrow><mo stretchy="false">(</mo>
<mi mathvariant='italic'>j</mi>
<mo stretchy="false">)</mo>
</mrow>
</msup>
</mrow></math>
above is a shorthand for
<math xmlns="http://www.w3.org/1998/Math/MathML" display="block"><mrow>
<msup><mi mathvariant='italic'>z</mi>
<mrow><mo stretchy="false">(</mo>
<mi mathvariant='italic'>j</mi>
<mo stretchy="false">)</mo>
</mrow>
</msup>
<mo stretchy="false">(</mo>
<msup><mi mathvariant='italic'>z</mi>
<mrow><mo stretchy="false">(</mo>
<mi mathvariant='italic'>j</mi>
<mn>-1</mn>
<mo stretchy="false">)</mo>
</mrow>
</msup>
<mo stretchy="false">,</mo>
<mo stretchy="false">…</mo>
<mo stretchy="false">,</mo>
<msup><mi mathvariant='italic'>z</mi>
<mrow><mo stretchy="false">(</mo>
<mn>0</mn>
<mo stretchy="false">)</mo>
</mrow>
</msup>
<mo stretchy="false">,</mo>
<mi mathvariant='italic'>x</mi>
<mo stretchy="false">,</mo>
<mi mathvariant='italic'>y</mi>
<mo stretchy="false">)</mo>
</mrow></math>
If we do not provide a formula for
a partial derivative of
<math xmlns="http://www.w3.org/1998/Math/MathML" display="inline"><mrow>
<mi mathvariant='italic'>H</mi>
</mrow></math>
, then that partial derivative
has the same value as for the function
<math xmlns="http://www.w3.org/1998/Math/MathML" display="inline"><mrow>
<mi mathvariant='italic'>G</mi>
</mrow></math>
.
<br/>
<br/>
<b><big><a name="Binary Operators" id="Binary Operators">Binary Operators</a></big></b>
<br/>
<br/>
<b><a name="Binary Operators.Addition" id="Binary Operators.Addition">Addition</a></b>
<br/>
The forward mode formula for
<a href="forwardtheory.xml#Binary Operators.Addition" target="_top"><span style='white-space: nowrap'>addition</span></a>
is
<math xmlns="http://www.w3.org/1998/Math/MathML" display="block"><mrow>
<msup><mi mathvariant='italic'>z</mi>
<mrow><mo stretchy="false">(</mo>
<mi mathvariant='italic'>j</mi>
<mo stretchy="false">)</mo>
</mrow>
</msup>
<mo stretchy="false">=</mo>
<msup><mi mathvariant='italic'>x</mi>
<mrow><mo stretchy="false">(</mo>
<mi mathvariant='italic'>j</mi>
<mo stretchy="false">)</mo>
</mrow>
</msup>
<mo stretchy="false">+</mo>
<msup><mi mathvariant='italic'>y</mi>
<mrow><mo stretchy="false">(</mo>
<mi mathvariant='italic'>j</mi>
<mo stretchy="false">)</mo>
</mrow>
</msup>
</mrow></math>
If follows that for
<math xmlns="http://www.w3.org/1998/Math/MathML" display="inline"><mrow>
<mi mathvariant='italic'>k</mi>
<mo stretchy="false">=</mo>
<mn>0</mn>
<mo stretchy="false">,</mo>
<mo stretchy="false">…</mo>
<mo stretchy="false">,</mo>
<mi mathvariant='italic'>j</mi>
</mrow></math>
and
<math xmlns="http://www.w3.org/1998/Math/MathML" display="inline"><mrow>
<mi mathvariant='italic'>l</mi>
<mo stretchy="false">=</mo>
<mn>0</mn>
<mo stretchy="false">,</mo>
<mo stretchy="false">…</mo>
<mo stretchy="false">,</mo>
<mi mathvariant='italic'>j</mi>
<mn>-1</mn>
</mrow></math>
<math xmlns="http://www.w3.org/1998/Math/MathML" display="block"><mrow>
<mtable rowalign="center" ><mtr><mtd columnalign="right" >
<mfrac><mrow><mo stretchy="false">∂</mo>
<mi mathvariant='italic'>H</mi>
</mrow>
<mrow><mo stretchy="false">∂</mo>
<msup><mi mathvariant='italic'>x</mi>
<mrow><mo stretchy="false">(</mo>
<mi mathvariant='italic'>k</mi>
<mo stretchy="false">)</mo>
</mrow>
</msup>
</mrow>
</mfrac>
</mtd><mtd columnalign="center" >
<mo stretchy="false">=</mo>
</mtd><mtd columnalign="left" >
<mfrac><mrow><mo stretchy="false">∂</mo>
<mi mathvariant='italic'>G</mi>
</mrow>
<mrow><mo stretchy="false">∂</mo>
<msup><mi mathvariant='italic'>x</mi>
<mrow><mo stretchy="false">(</mo>
<mi mathvariant='italic'>k</mi>
<mo stretchy="false">)</mo>
</mrow>
</msup>
</mrow>
</mfrac>
<mo stretchy="false">+</mo>
<mfrac><mrow><mo stretchy="false">∂</mo>
<mi mathvariant='italic'>G</mi>
</mrow>
<mrow><mo stretchy="false">∂</mo>
<msup><mi mathvariant='italic'>z</mi>
<mrow><mo stretchy="false">(</mo>
<mi mathvariant='italic'>k</mi>
<mo stretchy="false">)</mo>
</mrow>
</msup>
</mrow>
</mfrac>
</mtd></mtr><mtr><mtd columnalign="right" >
</mtd></mtr><mtr><mtd columnalign="right" >
<mfrac><mrow><mo stretchy="false">∂</mo>
<mi mathvariant='italic'>H</mi>
</mrow>
<mrow><mo stretchy="false">∂</mo>
<msup><mi mathvariant='italic'>y</mi>
<mrow><mo stretchy="false">(</mo>
<mi mathvariant='italic'>k</mi>
<mo stretchy="false">)</mo>
</mrow>
</msup>
</mrow>
</mfrac>
</mtd><mtd columnalign="center" >
<mo stretchy="false">=</mo>
</mtd><mtd columnalign="left" >
<mfrac><mrow><mo stretchy="false">∂</mo>
<mi mathvariant='italic'>G</mi>
</mrow>
<mrow><mo stretchy="false">∂</mo>
<msup><mi mathvariant='italic'>y</mi>
<mrow><mo stretchy="false">(</mo>
<mi mathvariant='italic'>k</mi>
<mo stretchy="false">)</mo>
</mrow>
</msup>
</mrow>
</mfrac>
<mo stretchy="false">+</mo>
<mfrac><mrow><mo stretchy="false">∂</mo>
<mi mathvariant='italic'>G</mi>
</mrow>
<mrow><mo stretchy="false">∂</mo>
<msup><mi mathvariant='italic'>z</mi>
<mrow><mo stretchy="false">(</mo>
<mi mathvariant='italic'>k</mi>
<mo stretchy="false">)</mo>
</mrow>
</msup>
</mrow>
</mfrac>
</mtd></mtr><mtr><mtd columnalign="right" >
<mfrac><mrow><mo stretchy="false">∂</mo>
<mi mathvariant='italic'>H</mi>
</mrow>
<mrow><mo stretchy="false">∂</mo>
<msup><mi mathvariant='italic'>z</mi>
<mrow><mo stretchy="false">(</mo>
<mi mathvariant='italic'>l</mi>
<mo stretchy="false">)</mo>
</mrow>
</msup>
</mrow>
</mfrac>
</mtd><mtd columnalign="center" >
<mo stretchy="false">=</mo>
</mtd><mtd columnalign="left" >
<mfrac><mrow><mo stretchy="false">∂</mo>
<mi mathvariant='italic'>G</mi>
</mrow>
<mrow><mo stretchy="false">∂</mo>
<msup><mi mathvariant='italic'>z</mi>
<mrow><mo stretchy="false">(</mo>
<mi mathvariant='italic'>l</mi>
<mo stretchy="false">)</mo>
</mrow>
</msup>
</mrow>
</mfrac>
</mtd></mtr></mtable>
</mrow></math>
<br/>
<b><a name="Binary Operators.Subtraction" id="Binary Operators.Subtraction">Subtraction</a></b>
<br/>
The forward mode formula for
<a href="forwardtheory.xml#Binary Operators.Subtraction" target="_top"><span style='white-space: nowrap'>subtraction</span></a>
is
<math xmlns="http://www.w3.org/1998/Math/MathML" display="block"><mrow>
<msup><mi mathvariant='italic'>z</mi>
<mrow><mo stretchy="false">(</mo>
<mi mathvariant='italic'>j</mi>
<mo stretchy="false">)</mo>
</mrow>
</msup>
<mo stretchy="false">=</mo>
<msup><mi mathvariant='italic'>x</mi>
<mrow><mo stretchy="false">(</mo>
<mi mathvariant='italic'>j</mi>
<mo stretchy="false">)</mo>
</mrow>
</msup>
<mo stretchy="false">-</mo>
<msup><mi mathvariant='italic'>y</mi>
<mrow><mo stretchy="false">(</mo>
<mi mathvariant='italic'>j</mi>
<mo stretchy="false">)</mo>
</mrow>
</msup>
</mrow></math>
If follows that for
<math xmlns="http://www.w3.org/1998/Math/MathML" display="inline"><mrow>
<mi mathvariant='italic'>k</mi>
<mo stretchy="false">=</mo>
<mn>0</mn>
<mo stretchy="false">,</mo>
<mo stretchy="false">…</mo>
<mo stretchy="false">,</mo>
<mi mathvariant='italic'>j</mi>
</mrow></math>
<math xmlns="http://www.w3.org/1998/Math/MathML" display="block"><mrow>
<mtable rowalign="center" ><mtr><mtd columnalign="right" >
<mfrac><mrow><mo stretchy="false">∂</mo>
<mi mathvariant='italic'>H</mi>
</mrow>
<mrow><mo stretchy="false">∂</mo>
<msup><mi mathvariant='italic'>x</mi>
<mrow><mo stretchy="false">(</mo>
<mi mathvariant='italic'>k</mi>
<mo stretchy="false">)</mo>
</mrow>
</msup>
</mrow>
</mfrac>
</mtd><mtd columnalign="center" >
<mo stretchy="false">=</mo>
</mtd><mtd columnalign="left" >
<mfrac><mrow><mo stretchy="false">∂</mo>
<mi mathvariant='italic'>G</mi>
</mrow>
<mrow><mo stretchy="false">∂</mo>
<msup><mi mathvariant='italic'>x</mi>
<mrow><mo stretchy="false">(</mo>
<mi mathvariant='italic'>k</mi>
<mo stretchy="false">)</mo>
</mrow>
</msup>
</mrow>
</mfrac>
<mo stretchy="false">-</mo>
<mfrac><mrow><mo stretchy="false">∂</mo>
<mi mathvariant='italic'>G</mi>
</mrow>
<mrow><mo stretchy="false">∂</mo>
<msup><mi mathvariant='italic'>z</mi>
<mrow><mo stretchy="false">(</mo>
<mi mathvariant='italic'>k</mi>
<mo stretchy="false">)</mo>
</mrow>
</msup>
</mrow>
</mfrac>
</mtd></mtr><mtr><mtd columnalign="right" >
</mtd></mtr><mtr><mtd columnalign="right" >
<mfrac><mrow><mo stretchy="false">∂</mo>
<mi mathvariant='italic'>H</mi>
</mrow>
<mrow><mo stretchy="false">∂</mo>
<msup><mi mathvariant='italic'>y</mi>
<mrow><mo stretchy="false">(</mo>
<mi mathvariant='italic'>k</mi>
<mo stretchy="false">)</mo>
</mrow>
</msup>
</mrow>
</mfrac>
</mtd><mtd columnalign="center" >
<mo stretchy="false">=</mo>
</mtd><mtd columnalign="left" >
<mfrac><mrow><mo stretchy="false">∂</mo>
<mi mathvariant='italic'>G</mi>
</mrow>
<mrow><mo stretchy="false">∂</mo>
<msup><mi mathvariant='italic'>y</mi>
<mrow><mo stretchy="false">(</mo>
<mi mathvariant='italic'>k</mi>
<mo stretchy="false">)</mo>
</mrow>
</msup>
</mrow>
</mfrac>
<mo stretchy="false">-</mo>
<mfrac><mrow><mo stretchy="false">∂</mo>
<mi mathvariant='italic'>G</mi>
</mrow>
<mrow><mo stretchy="false">∂</mo>
<msup><mi mathvariant='italic'>z</mi>
<mrow><mo stretchy="false">(</mo>
<mi mathvariant='italic'>k</mi>
<mo stretchy="false">)</mo>
</mrow>
</msup>
</mrow>
</mfrac>
</mtd></mtr></mtable>
</mrow></math>
<br/>
<b><a name="Binary Operators.Multiplication" id="Binary Operators.Multiplication">Multiplication</a></b>
<br/>
The forward mode formula for
<a href="forwardtheory.xml#Binary Operators.Multiplication" target="_top"><span style='white-space: nowrap'>multiplication</span></a>
is
<math xmlns="http://www.w3.org/1998/Math/MathML" display="block"><mrow>
<msup><mi mathvariant='italic'>z</mi>
<mrow><mo stretchy="false">(</mo>
<mi mathvariant='italic'>j</mi>
<mo stretchy="false">)</mo>
</mrow>
</msup>
<mo stretchy="false">=</mo>
<munderover><mo displaystyle='true' largeop='true'>∑</mo>
<mrow><mi mathvariant='italic'>k</mi>
<mo stretchy="false">=</mo>
<mn>0</mn>
</mrow>
<mi mathvariant='italic'>j</mi>
</munderover>
<msup><mi mathvariant='italic'>x</mi>
<mrow><mo stretchy="false">(</mo>
<mi mathvariant='italic'>j</mi>
<mo stretchy="false">-</mo>
<mi mathvariant='italic'>k</mi>
<mo stretchy="false">)</mo>
</mrow>
</msup>
<mo stretchy="false">*</mo>
<msup><mi mathvariant='italic'>y</mi>
<mrow><mo stretchy="false">(</mo>
<mi mathvariant='italic'>k</mi>
<mo stretchy="false">)</mo>
</mrow>
</msup>
</mrow></math>
If follows that for
<math xmlns="http://www.w3.org/1998/Math/MathML" display="inline"><mrow>
<mi mathvariant='italic'>k</mi>
<mo stretchy="false">=</mo>
<mn>0</mn>
<mo stretchy="false">,</mo>
<mo stretchy="false">…</mo>
<mo stretchy="false">,</mo>
<mi mathvariant='italic'>j</mi>
</mrow></math>
and
<math xmlns="http://www.w3.org/1998/Math/MathML" display="inline"><mrow>
<mi mathvariant='italic'>l</mi>
<mo stretchy="false">=</mo>
<mn>0</mn>
<mo stretchy="false">,</mo>
<mo stretchy="false">…</mo>
<mo stretchy="false">,</mo>
<mi mathvariant='italic'>j</mi>
<mn>-1</mn>
</mrow></math>
<math xmlns="http://www.w3.org/1998/Math/MathML" display="block"><mrow>
<mtable rowalign="center" ><mtr><mtd columnalign="right" >
<mfrac><mrow><mo stretchy="false">∂</mo>
<mi mathvariant='italic'>H</mi>
</mrow>
<mrow><mo stretchy="false">∂</mo>
<msup><mi mathvariant='italic'>x</mi>
<mrow><mo stretchy="false">(</mo>
<mi mathvariant='italic'>j</mi>
<mo stretchy="false">-</mo>
<mi mathvariant='italic'>k</mi>
<mo stretchy="false">)</mo>
</mrow>
</msup>
</mrow>
</mfrac>
</mtd><mtd columnalign="center" >
<mo stretchy="false">=</mo>
</mtd><mtd columnalign="left" >
<mfrac><mrow><mo stretchy="false">∂</mo>
<mi mathvariant='italic'>G</mi>
</mrow>
<mrow><mo stretchy="false">∂</mo>
<msup><mi mathvariant='italic'>x</mi>
<mrow><mo stretchy="false">(</mo>
<mi mathvariant='italic'>j</mi>
<mo stretchy="false">-</mo>
<mi mathvariant='italic'>k</mi>
<mo stretchy="false">)</mo>
</mrow>
</msup>
</mrow>
</mfrac>
<mo stretchy="false">+</mo>
<munderover><mo displaystyle='true' largeop='true'>∑</mo>
<mrow><mi mathvariant='italic'>k</mi>
<mo stretchy="false">=</mo>
<mn>0</mn>
</mrow>
<mi mathvariant='italic'>j</mi>
</munderover>
<mfrac><mrow><mo stretchy="false">∂</mo>
<mi mathvariant='italic'>G</mi>
</mrow>
<mrow><mo stretchy="false">∂</mo>
<msup><mi mathvariant='italic'>z</mi>
<mrow><mo stretchy="false">(</mo>
<mi mathvariant='italic'>j</mi>
<mo stretchy="false">)</mo>
</mrow>
</msup>
</mrow>
</mfrac>
<msup><mi mathvariant='italic'>y</mi>
<mrow><mo stretchy="false">(</mo>
<mi mathvariant='italic'>k</mi>
<mo stretchy="false">)</mo>
</mrow>
</msup>
</mtd></mtr><mtr><mtd columnalign="right" >
<mfrac><mrow><mo stretchy="false">∂</mo>
<mi mathvariant='italic'>H</mi>
</mrow>
<mrow><mo stretchy="false">∂</mo>
<msup><mi mathvariant='italic'>y</mi>
<mrow><mo stretchy="false">(</mo>
<mi mathvariant='italic'>k</mi>
<mo stretchy="false">)</mo>
</mrow>
</msup>
</mrow>
</mfrac>
</mtd><mtd columnalign="center" >
<mo stretchy="false">=</mo>
</mtd><mtd columnalign="left" >
<mfrac><mrow><mo stretchy="false">∂</mo>
<mi mathvariant='italic'>G</mi>
</mrow>
<mrow><mo stretchy="false">∂</mo>
<msup><mi mathvariant='italic'>y</mi>
<mrow><mo stretchy="false">(</mo>
<mi mathvariant='italic'>k</mi>
<mo stretchy="false">)</mo>
</mrow>
</msup>
</mrow>
</mfrac>
<mo stretchy="false">+</mo>
<munderover><mo displaystyle='true' largeop='true'>∑</mo>
<mrow><mi mathvariant='italic'>k</mi>
<mo stretchy="false">=</mo>
<mn>0</mn>
</mrow>
<mi mathvariant='italic'>j</mi>
</munderover>
<mfrac><mrow><mo stretchy="false">∂</mo>
<mi mathvariant='italic'>G</mi>
</mrow>
<mrow><mo stretchy="false">∂</mo>
<msup><mi mathvariant='italic'>z</mi>
<mrow><mo stretchy="false">(</mo>
<mi mathvariant='italic'>j</mi>
<mo stretchy="false">)</mo>
</mrow>
</msup>
</mrow>
</mfrac>
<msup><mi mathvariant='italic'>x</mi>
<mrow><mo stretchy="false">(</mo>
<mi mathvariant='italic'>j</mi>
<mo stretchy="false">-</mo>
<mi mathvariant='italic'>k</mi>
<mo stretchy="false">)</mo>
</mrow>
</msup>
</mtd></mtr></mtable>
</mrow></math>
<br/>
<b><a name="Binary Operators.Division" id="Binary Operators.Division">Division</a></b>
<br/>
The forward mode formula for
<a href="forwardtheory.xml#Binary Operators.Division" target="_top"><span style='white-space: nowrap'>division</span></a>
is
<math xmlns="http://www.w3.org/1998/Math/MathML" display="block"><mrow>
<msup><mi mathvariant='italic'>z</mi>
<mrow><mo stretchy="false">(</mo>
<mi mathvariant='italic'>j</mi>
<mo stretchy="false">)</mo>
</mrow>
</msup>
<mo stretchy="false">=</mo>
<mfrac><mrow><mn>1</mn>
</mrow>
<mrow><msup><mi mathvariant='italic'>y</mi>
<mrow><mo stretchy="false">(</mo>
<mn>0</mn>
<mo stretchy="false">)</mo>
</mrow>
</msup>
</mrow>
</mfrac>
<mrow><mo stretchy="true">(</mo><mrow><msup><mi mathvariant='italic'>x</mi>
<mrow><mo stretchy="false">(</mo>
<mi mathvariant='italic'>j</mi>
<mo stretchy="false">)</mo>
</mrow>
</msup>
<mo stretchy="false">-</mo>
<munderover><mo displaystyle='true' largeop='true'>∑</mo>
<mrow><mi mathvariant='italic'>k</mi>
<mo stretchy="false">=</mo>
<mn>1</mn>
</mrow>
<mi mathvariant='italic'>j</mi>
</munderover>
<msup><mi mathvariant='italic'>z</mi>
<mrow><mo stretchy="false">(</mo>
<mi mathvariant='italic'>j</mi>
<mo stretchy="false">-</mo>
<mi mathvariant='italic'>k</mi>
<mo stretchy="false">)</mo>
</mrow>
</msup>
<msup><mi mathvariant='italic'>y</mi>
<mrow><mo stretchy="false">(</mo>
<mi mathvariant='italic'>k</mi>
<mo stretchy="false">)</mo>
</mrow>
</msup>
</mrow><mo stretchy="true">)</mo></mrow>
</mrow></math>
If follows that for
<math xmlns="http://www.w3.org/1998/Math/MathML" display="inline"><mrow>
<mi mathvariant='italic'>k</mi>
<mo stretchy="false">=</mo>
<mn>1</mn>
<mo stretchy="false">,</mo>
<mo stretchy="false">…</mo>
<mo stretchy="false">,</mo>
<mi mathvariant='italic'>j</mi>
</mrow></math>
<math xmlns="http://www.w3.org/1998/Math/MathML" display="block"><mrow>
<mtable rowalign="center" ><mtr><mtd columnalign="right" >
<mfrac><mrow><mo stretchy="false">∂</mo>
<mi mathvariant='italic'>H</mi>
</mrow>
<mrow><mo stretchy="false">∂</mo>
<msup><mi mathvariant='italic'>x</mi>
<mrow><mo stretchy="false">(</mo>
<mi mathvariant='italic'>j</mi>
<mo stretchy="false">)</mo>
</mrow>
</msup>
</mrow>
</mfrac>
</mtd><mtd columnalign="center" >
<mo stretchy="false">=</mo>
</mtd><mtd columnalign="left" >
<mfrac><mrow><mo stretchy="false">∂</mo>
<mi mathvariant='italic'>G</mi>
</mrow>
<mrow><mo stretchy="false">∂</mo>
<msup><mi mathvariant='italic'>x</mi>
<mrow><mo stretchy="false">(</mo>
<mi mathvariant='italic'>j</mi>
<mo stretchy="false">)</mo>
</mrow>
</msup>
</mrow>
</mfrac>
<mo stretchy="false">+</mo>
<mfrac><mrow><mo stretchy="false">∂</mo>
<mi mathvariant='italic'>G</mi>
</mrow>
<mrow><mo stretchy="false">∂</mo>
<msup><mi mathvariant='italic'>z</mi>
<mrow><mo stretchy="false">(</mo>
<mi mathvariant='italic'>j</mi>
<mo stretchy="false">)</mo>
</mrow>
</msup>
</mrow>
</mfrac>
<mfrac><mrow><mn>1</mn>
</mrow>
<mrow><msup><mi mathvariant='italic'>y</mi>
<mrow><mo stretchy="false">(</mo>
<mn>0</mn>
<mo stretchy="false">)</mo>
</mrow>
</msup>
</mrow>
</mfrac>
</mtd></mtr><mtr><mtd columnalign="right" >
<mfrac><mrow><mo stretchy="false">∂</mo>
<mi mathvariant='italic'>H</mi>
</mrow>
<mrow><mo stretchy="false">∂</mo>
<msup><mi mathvariant='italic'>z</mi>
<mrow><mo stretchy="false">(</mo>
<mi mathvariant='italic'>j</mi>
<mo stretchy="false">-</mo>
<mi mathvariant='italic'>k</mi>
<mo stretchy="false">)</mo>
</mrow>
</msup>
</mrow>
</mfrac>
</mtd><mtd columnalign="center" >
<mo stretchy="false">=</mo>
</mtd><mtd columnalign="left" >
<mfrac><mrow><mo stretchy="false">∂</mo>
<mi mathvariant='italic'>G</mi>
</mrow>
<mrow><mo stretchy="false">∂</mo>
<msup><mi mathvariant='italic'>z</mi>
<mrow><mo stretchy="false">(</mo>
<mi mathvariant='italic'>j</mi>
<mo stretchy="false">-</mo>
<mi mathvariant='italic'>k</mi>
<mo stretchy="false">)</mo>
</mrow>
</msup>
</mrow>
</mfrac>
<mo stretchy="false">-</mo>
<mfrac><mrow><mo stretchy="false">∂</mo>
<mi mathvariant='italic'>G</mi>
</mrow>
<mrow><mo stretchy="false">∂</mo>
<msup><mi mathvariant='italic'>z</mi>
<mrow><mo stretchy="false">(</mo>
<mi mathvariant='italic'>j</mi>
<mo stretchy="false">)</mo>
</mrow>
</msup>
</mrow>
</mfrac>
<mfrac><mrow><mn>1</mn>
</mrow>
<mrow><msup><mi mathvariant='italic'>y</mi>
<mrow><mo stretchy="false">(</mo>
<mn>0</mn>
<mo stretchy="false">)</mo>
</mrow>
</msup>
</mrow>
</mfrac>
<msup><mi mathvariant='italic'>y</mi>
<mrow><mo stretchy="false">(</mo>
<mi mathvariant='italic'>k</mi>
<mo stretchy="false">)</mo>
</mrow>
</msup>
</mtd></mtr><mtr><mtd columnalign="right" >
<mfrac><mrow><mo stretchy="false">∂</mo>
<mi mathvariant='italic'>H</mi>
</mrow>
<mrow><mo stretchy="false">∂</mo>
<msup><mi mathvariant='italic'>y</mi>
<mrow><mo stretchy="false">(</mo>
<mi mathvariant='italic'>k</mi>
<mo stretchy="false">)</mo>
</mrow>
</msup>
</mrow>
</mfrac>
</mtd><mtd columnalign="center" >
<mo stretchy="false">=</mo>
</mtd><mtd columnalign="left" >
<mfrac><mrow><mo stretchy="false">∂</mo>
<mi mathvariant='italic'>G</mi>
</mrow>
<mrow><mo stretchy="false">∂</mo>
<msup><mi mathvariant='italic'>y</mi>
<mrow><mo stretchy="false">(</mo>
<mi mathvariant='italic'>k</mi>
<mo stretchy="false">)</mo>
</mrow>
</msup>
</mrow>
</mfrac>
<mo stretchy="false">-</mo>
<mfrac><mrow><mo stretchy="false">∂</mo>
<mi mathvariant='italic'>G</mi>
</mrow>
<mrow><mo stretchy="false">∂</mo>
<msup><mi mathvariant='italic'>z</mi>
<mrow><mo stretchy="false">(</mo>
<mi mathvariant='italic'>j</mi>
<mo stretchy="false">)</mo>
</mrow>
</msup>
</mrow>
</mfrac>
<mfrac><mrow><mn>1</mn>
</mrow>
<mrow><msup><mi mathvariant='italic'>y</mi>
<mrow><mo stretchy="false">(</mo>
<mn>0</mn>
<mo stretchy="false">)</mo>
</mrow>
</msup>
</mrow>
</mfrac>
<msup><mi mathvariant='italic'>z</mi>
<mrow><mo stretchy="false">(</mo>
<mi mathvariant='italic'>j</mi>
<mo stretchy="false">-</mo>
<mi mathvariant='italic'>k</mi>
<mo stretchy="false">)</mo>
</mrow>
</msup>
</mtd></mtr><mtr><mtd columnalign="right" >
<mfrac><mrow><mo stretchy="false">∂</mo>
<mi mathvariant='italic'>H</mi>
</mrow>
<mrow><mo stretchy="false">∂</mo>
<msup><mi mathvariant='italic'>y</mi>
<mrow><mo stretchy="false">(</mo>
<mn>0</mn>
<mo stretchy="false">)</mo>
</mrow>
</msup>
</mrow>
</mfrac>
</mtd><mtd columnalign="center" >
<mo stretchy="false">=</mo>
</mtd><mtd columnalign="left" >
<mfrac><mrow><mo stretchy="false">∂</mo>
<mi mathvariant='italic'>G</mi>
</mrow>
<mrow><mo stretchy="false">∂</mo>
<msup><mi mathvariant='italic'>y</mi>
<mrow><mo stretchy="false">(</mo>
<mn>0</mn>
<mo stretchy="false">)</mo>
</mrow>
</msup>
</mrow>
</mfrac>
<mo stretchy="false">-</mo>
<mfrac><mrow><mo stretchy="false">∂</mo>
<mi mathvariant='italic'>G</mi>
</mrow>
<mrow><mo stretchy="false">∂</mo>
<msup><mi mathvariant='italic'>z</mi>
<mrow><mo stretchy="false">(</mo>
<mi mathvariant='italic'>j</mi>
<mo stretchy="false">)</mo>
</mrow>
</msup>
</mrow>
</mfrac>
<mfrac><mrow><mn>1</mn>
</mrow>
<mrow><msup><mi mathvariant='italic'>y</mi>
<mrow><mo stretchy="false">(</mo>
<mn>0</mn>
<mo stretchy="false">)</mo>
</mrow>
</msup>
</mrow>
</mfrac>
<mfrac><mrow><mn>1</mn>
</mrow>
<mrow><msup><mi mathvariant='italic'>y</mi>
<mrow><mo stretchy="false">(</mo>
<mn>0</mn>
<mo stretchy="false">)</mo>
</mrow>
</msup>
</mrow>
</mfrac>
<mrow><mo stretchy="true">(</mo><mrow><msup><mi mathvariant='italic'>x</mi>
<mrow><mo stretchy="false">(</mo>
<mi mathvariant='italic'>j</mi>
<mo stretchy="false">)</mo>
</mrow>
</msup>
<mo stretchy="false">-</mo>
<munderover><mo displaystyle='true' largeop='true'>∑</mo>
<mrow><mi mathvariant='italic'>k</mi>
<mo stretchy="false">=</mo>
<mn>1</mn>
</mrow>
<mi mathvariant='italic'>j</mi>
</munderover>
<msup><mi mathvariant='italic'>z</mi>
<mrow><mo stretchy="false">(</mo>
<mi mathvariant='italic'>j</mi>
<mo stretchy="false">-</mo>
<mi mathvariant='italic'>k</mi>
<mo stretchy="false">)</mo>
</mrow>
</msup>
<msup><mi mathvariant='italic'>y</mi>
<mrow><mo stretchy="false">(</mo>
<mi mathvariant='italic'>k</mi>
<mo stretchy="false">)</mo>
</mrow>
</msup>
</mrow><mo stretchy="true">)</mo></mrow>
</mtd></mtr><mtr><mtd columnalign="right" >
</mtd><mtd columnalign="center" >
<mo stretchy="false">=</mo>
</mtd><mtd columnalign="left" >
<mfrac><mrow><mo stretchy="false">∂</mo>
<mi mathvariant='italic'>G</mi>
</mrow>
<mrow><mo stretchy="false">∂</mo>
<msup><mi mathvariant='italic'>y</mi>
<mrow><mo stretchy="false">(</mo>
<mn>0</mn>
<mo stretchy="false">)</mo>
</mrow>
</msup>
</mrow>
</mfrac>
<mo stretchy="false">-</mo>
<mfrac><mrow><mo stretchy="false">∂</mo>
<mi mathvariant='italic'>G</mi>
</mrow>
<mrow><mo stretchy="false">∂</mo>
<msup><mi mathvariant='italic'>z</mi>
<mrow><mo stretchy="false">(</mo>
<mi mathvariant='italic'>j</mi>
<mo stretchy="false">)</mo>
</mrow>
</msup>
</mrow>
</mfrac>
<mfrac><mrow><mn>1</mn>
</mrow>
<mrow><msup><mi mathvariant='italic'>y</mi>
<mrow><mo stretchy="false">(</mo>
<mn>0</mn>
<mo stretchy="false">)</mo>
</mrow>
</msup>
</mrow>
</mfrac>
<msup><mi mathvariant='italic'>z</mi>
<mrow><mo stretchy="false">(</mo>
<mi mathvariant='italic'>j</mi>
<mo stretchy="false">)</mo>
</mrow>
</msup>
</mtd></mtr></mtable>
</mrow></math>
<br/>
<b><big><a name="Standard Math Functions" id="Standard Math Functions">Standard Math Functions</a></big></b>
<br/>
The standard math functions have only one argument.
Hence we are given the partial derivatives of a scalar valued function
<math xmlns="http://www.w3.org/1998/Math/MathML" display="block"><mrow>
<mi mathvariant='italic'>G</mi>
<mo stretchy="false">(</mo>
<msup><mi mathvariant='italic'>z</mi>
<mrow><mo stretchy="false">(</mo>
<mi mathvariant='italic'>j</mi>
<mo stretchy="false">)</mo>
</mrow>
</msup>
<mo stretchy="false">,</mo>
<mo stretchy="false">…</mo>
<mo stretchy="false">,</mo>
<msup><mi mathvariant='italic'>z</mi>
<mrow><mo stretchy="false">(</mo>
<mn>0</mn>
<mo stretchy="false">)</mo>
</mrow>
</msup>
<mo stretchy="false">,</mo>
<mi mathvariant='italic'>x</mi>
<mo stretchy="false">)</mo>
</mrow></math>
We need to compute the partial derivatives of the scalar valued function
<math xmlns="http://www.w3.org/1998/Math/MathML" display="block"><mrow>
<mi mathvariant='italic'>H</mi>
<mo stretchy="false">(</mo>
<msup><mi mathvariant='italic'>z</mi>
<mrow><mo stretchy="false">(</mo>
<mi mathvariant='italic'>j</mi>
<mn>-1</mn>
<mo stretchy="false">)</mo>
</mrow>
</msup>
<mo stretchy="false">,</mo>
<mo stretchy="false">…</mo>
<mo stretchy="false">,</mo>
<msup><mi mathvariant='italic'>z</mi>
<mrow><mo stretchy="false">(</mo>
<mn>0</mn>
<mo stretchy="false">)</mo>
</mrow>
</msup>
<mo stretchy="false">,</mo>
<mi mathvariant='italic'>x</mi>
<mo stretchy="false">)</mo>
<mo stretchy="false">=</mo>
<mi mathvariant='italic'>G</mi>
<mo stretchy="false">(</mo>
<msup><mi mathvariant='italic'>z</mi>
<mrow><mo stretchy="false">(</mo>
<mi mathvariant='italic'>j</mi>
<mo stretchy="false">)</mo>
</mrow>
</msup>
<mo stretchy="false">,</mo>
<msup><mi mathvariant='italic'>z</mi>
<mrow><mo stretchy="false">(</mo>
<mi mathvariant='italic'>j</mi>
<mn>-1</mn>
<mo stretchy="false">)</mo>
</mrow>
</msup>
<mo stretchy="false">,</mo>
<mo stretchy="false">…</mo>
<mo stretchy="false">,</mo>
<msup><mi mathvariant='italic'>z</mi>
<mrow><mo stretchy="false">(</mo>
<mn>0</mn>
<mo stretchy="false">)</mo>
</mrow>
</msup>
<mo stretchy="false">,</mo>
<mi mathvariant='italic'>x</mi>
<mo stretchy="false">)</mo>
</mrow></math>
where
<math xmlns="http://www.w3.org/1998/Math/MathML" display="inline"><mrow>
<msup><mi mathvariant='italic'>z</mi>
<mrow><mo stretchy="false">(</mo>
<mi mathvariant='italic'>j</mi>
<mo stretchy="false">)</mo>
</mrow>
</msup>
</mrow></math>
is expressed as a function of the
<code><i>j-1</i></code>-th order Taylor coefficient row
vector for
<math xmlns="http://www.w3.org/1998/Math/MathML" display="inline"><mrow>
<mi mathvariant='italic'>Z</mi>
</mrow></math>
and the vector
<math xmlns="http://www.w3.org/1998/Math/MathML" display="inline"><mrow>
<mi mathvariant='italic'>x</mi>
</mrow></math>
; i.e.,
<math xmlns="http://www.w3.org/1998/Math/MathML" display="inline"><mrow>
<msup><mi mathvariant='italic'>z</mi>
<mrow><mo stretchy="false">(</mo>
<mi mathvariant='italic'>j</mi>
<mo stretchy="false">)</mo>
</mrow>
</msup>
</mrow></math>
above is a shorthand for
<math xmlns="http://www.w3.org/1998/Math/MathML" display="block"><mrow>
<msup><mi mathvariant='italic'>z</mi>
<mrow><mo stretchy="false">(</mo>
<mi mathvariant='italic'>j</mi>
<mo stretchy="false">)</mo>
</mrow>
</msup>
<mo stretchy="false">(</mo>
<msup><mi mathvariant='italic'>z</mi>
<mrow><mo stretchy="false">(</mo>
<mi mathvariant='italic'>j</mi>
<mn>-1</mn>
<mo stretchy="false">)</mo>
</mrow>
</msup>
<mo stretchy="false">,</mo>
<mo stretchy="false">…</mo>
<mo stretchy="false">,</mo>
<msup><mi mathvariant='italic'>z</mi>
<mrow><mo stretchy="false">(</mo>
<mn>0</mn>
<mo stretchy="false">)</mo>
</mrow>
</msup>
<mo stretchy="false">,</mo>
<mi mathvariant='italic'>x</mi>
<mo stretchy="false">)</mo>
</mrow></math>
<br/>
<b><big><a name="Contents" id="Contents">Contents</a></big></b>
<br/>
<table>
<tr><td><a href="expreverse.xml" target="_top">ExpReverse</a></td><td>Exponential Function Reverse Mode Theory</td></tr><tr><td><a href="logreverse.xml" target="_top">LogReverse</a></td><td>Logarithm Function Reverse Mode Theory</td></tr><tr><td><a href="sqrtreverse.xml" target="_top">SqrtReverse</a></td><td>Square Root Function Reverse Mode Theory</td></tr><tr><td><a href="sincosreverse.xml" target="_top">SinCosReverse</a></td><td>Trigonometric and Hyperbolic Sine and Cosine Reverse Theory</td></tr><tr><td><a href="atanreverse.xml" target="_top">AtanReverse</a></td><td>Arctangent Function Reverse Mode Theory</td></tr><tr><td><a href="asinreverse.xml" target="_top">AsinReverse</a></td><td>Arcsine Function Reverse Mode Theory</td></tr><tr><td><a href="acosreverse.xml" target="_top">AcosReverse</a></td><td>Arccosine Function Reverse Mode Theory</td></tr><tr><td><a href="tan_reverse.xml" target="_top">tan_reverse</a></td><td>Tangent and Hyperbolic Tangent Reverse Mode Theory</td></tr><tr><td><a href="erf_reverse.xml" target="_top">erf_reverse</a></td><td>Error Function Reverse Mode Theory</td></tr></table>
<hr/>Input File: omh/theory/reverse_theory.omh
</body>
</html>