<?xml version='1.0'?>
<html xmlns='http://www.w3.org/1999/xhtml'
xmlns:math='http://www.w3.org/1998/Math/MathML'
>
<head>
<title>Second Order Reverse Mode</title>
<meta http-equiv='Content-Type' content='text/html' charset='utf-8'/>
<meta name="description" id="description" content="Second Order Reverse Mode"/>
<meta name="keywords" id="keywords" content=" second order reverse mode derivative syntax purpose x^(k) w f dw first partials vector hessian times direction example "/>
<style type='text/css'>
body { color : black }
body { background-color : white }
A:link { color : blue }
A:visited { color : purple }
A:active { color : purple }
</style>
<script type='text/javascript' language='JavaScript' src='_reverse_two_xml.js'>
</script>
</head>
<body>
<table><tr>
<td>
<a href="http://www.coin-or.org/CppAD/" target="_top"><img border="0" src="_image.gif"/></a>
</td>
<td><a href="reverse_one.cpp.xml" target="_top">Prev</a>
</td><td><a href="reverse_two.cpp.xml" target="_top">Next</a>
</td><td>
<select onchange='choose_across0(this)'>
<option>Index-></option>
<option>contents</option>
<option>reference</option>
<option>index</option>
<option>search</option>
<option>external</option>
</select>
</td>
<td>
<select onchange='choose_up0(this)'>
<option>Up-></option>
<option>CppAD</option>
<option>ADFun</option>
<option>FunEval</option>
<option>Reverse</option>
<option>reverse_two</option>
</select>
</td>
<td>
<select onchange='choose_down3(this)'>
<option>ADFun-></option>
<option>Independent</option>
<option>FunConstruct</option>
<option>Dependent</option>
<option>abort_recording</option>
<option>seq_property</option>
<option>FunEval</option>
<option>Drivers</option>
<option>FunCheck</option>
<option>optimize</option>
<option>check_for_nan</option>
</select>
</td>
<td>
<select onchange='choose_down2(this)'>
<option>FunEval-></option>
<option>Forward</option>
<option>Reverse</option>
<option>Sparse</option>
</select>
</td>
<td>
<select onchange='choose_down1(this)'>
<option>Reverse-></option>
<option>reverse_one</option>
<option>reverse_two</option>
<option>reverse_any</option>
</select>
</td>
<td>
<select onchange='choose_down0(this)'>
<option>reverse_two-></option>
<option>reverse_two.cpp</option>
<option>hes_times_dir.cpp</option>
</select>
</td>
<td>
<select onchange='choose_current0(this)'>
<option>Headings-></option>
<option>Syntax</option>
<option>Purpose</option>
<option>x^(k)</option>
<option>W</option>
<option>f</option>
<option>w</option>
<option>dw</option>
<option>---..First Order Partials</option>
<option>---..Second Order Partials</option>
<option>Vector</option>
<option>Hessian Times Direction</option>
<option>Example</option>
</select>
</td>
</tr></table><br/>
<center><b><big><big>Second Order Reverse Mode</big></big></b></center>
<br/>
<b><big><a name="Syntax" id="Syntax">Syntax</a></big></b>
<br/>
<code><i><font color="black"><span style='white-space: nowrap'>dw</span></font></i><font color="blue"><span style='white-space: nowrap'> = </span></font><i><font color="black"><span style='white-space: nowrap'>f</span></font></i><font color="blue"><span style='white-space: nowrap'>.Reverse(2, </span></font><i><font color="black"><span style='white-space: nowrap'>w</span></font></i><font color="blue"><span style='white-space: nowrap'>)</span></font></code>
<br/>
<br/>
<b><big><a name="Purpose" id="Purpose">Purpose</a></big></b>
<br/>
We use
<math xmlns="http://www.w3.org/1998/Math/MathML" display="inline"><mrow>
<mi mathvariant='italic'>F</mi>
<mo stretchy="false">:</mo>
<msup><mi mathvariant='italic'>B</mi>
<mi mathvariant='italic'>n</mi>
</msup>
<mo stretchy="false">→</mo>
<msup><mi mathvariant='italic'>B</mi>
<mi mathvariant='italic'>m</mi>
</msup>
</mrow></math>
to denote the
<a href="glossary.xml#AD Function" target="_top"><span style='white-space: nowrap'>AD function</span></a>
corresponding to
<code><i><font color="black"><span style='white-space: nowrap'>f</span></font></i></code>
.
Reverse mode computes the derivative of the <a href="forward.xml" target="_top"><span style='white-space: nowrap'>Forward</span></a>
mode
<a href="glossary.xml#Taylor Coefficient" target="_top"><span style='white-space: nowrap'>Taylor coefficients</span></a>
with respect to the domain variable
<math xmlns="http://www.w3.org/1998/Math/MathML" display="inline"><mrow>
<mi mathvariant='italic'>x</mi>
</mrow></math>
.
<br/>
<br/>
<b><big><a name="x^(k)" id="x^(k)">x^(k)</a></big></b>
<br/>
For
<math xmlns="http://www.w3.org/1998/Math/MathML" display="inline"><mrow>
<mi mathvariant='italic'>k</mi>
<mo stretchy="false">=</mo>
<mn>0</mn>
<mo stretchy="false">,</mo>
<mn>1</mn>
</mrow></math>
,
the vector
<math xmlns="http://www.w3.org/1998/Math/MathML" display="inline"><mrow>
<msup><mi mathvariant='italic'>x</mi>
<mrow><mo stretchy="false">(</mo>
<mi mathvariant='italic'>k</mi>
<mo stretchy="false">)</mo>
</mrow>
</msup>
<mo stretchy="false">∈</mo>
<msup><mi mathvariant='italic'>B</mi>
<mi mathvariant='italic'>n</mi>
</msup>
</mrow></math>
is defined as the value of
<code><i><font color="black"><span style='white-space: nowrap'>x_k</span></font></i></code>
in the previous call (counting this call) of the form
<code><font color="blue"><span style='white-space: nowrap'><br/>
     </span></font><i><font color="black"><span style='white-space: nowrap'>f</span></font></i><font color="blue"><span style='white-space: nowrap'>.Forward(</span></font><i><font color="black"><span style='white-space: nowrap'>k</span></font></i><font color="blue"><span style='white-space: nowrap'>, </span></font><i><font color="black"><span style='white-space: nowrap'>x_k</span></font></i><font color="blue"><span style='white-space: nowrap'>)<br/>
</span></font></code>
If there is no previous call with
<math xmlns="http://www.w3.org/1998/Math/MathML" display="inline"><mrow>
<mi mathvariant='italic'>k</mi>
<mo stretchy="false">=</mo>
<mn>0</mn>
</mrow></math>
,
<math xmlns="http://www.w3.org/1998/Math/MathML" display="inline"><mrow>
<msup><mi mathvariant='italic'>x</mi>
<mrow><mo stretchy="false">(</mo>
<mn>0</mn>
<mo stretchy="false">)</mo>
</mrow>
</msup>
</mrow></math>
is the value of the independent variables when the
corresponding
AD of
<code><i><font color="black"><span style='white-space: nowrap'>Base</span></font></i></code>
<a href="glossary.xml#Operation.Sequence" target="_top"><span style='white-space: nowrap'>operation sequence</span></a>
was recorded.
<br/>
<br/>
<b><big><a name="W" id="W">W</a></big></b>
<br/>
The functions
<math xmlns="http://www.w3.org/1998/Math/MathML" display="inline"><mrow>
<msub><mi mathvariant='italic'>W</mi>
<mn>0</mn>
</msub>
<mo stretchy="false">:</mo>
<msup><mi mathvariant='italic'>B</mi>
<mi mathvariant='italic'>n</mi>
</msup>
<mo stretchy="false">→</mo>
<mi mathvariant='italic'>B</mi>
</mrow></math>
and
<math xmlns="http://www.w3.org/1998/Math/MathML" display="inline"><mrow>
<msub><mi mathvariant='italic'>W</mi>
<mn>1</mn>
</msub>
<mo stretchy="false">:</mo>
<msup><mi mathvariant='italic'>B</mi>
<mi mathvariant='italic'>n</mi>
</msup>
<mo stretchy="false">→</mo>
<mi mathvariant='italic'>B</mi>
</mrow></math>
are defined by
<math xmlns="http://www.w3.org/1998/Math/MathML" display="block"><mrow>
<mtable rowalign="center" ><mtr><mtd columnalign="right" >
<msub><mi mathvariant='italic'>W</mi>
<mn>0</mn>
</msub>
<mo stretchy="false">(</mo>
<mi mathvariant='italic'>u</mi>
<mo stretchy="false">)</mo>
</mtd><mtd columnalign="center" >
<mo stretchy="false">=</mo>
</mtd><mtd columnalign="left" >
<msub><mi mathvariant='italic'>w</mi>
<mn>0</mn>
</msub>
<mo stretchy="false">*</mo>
<msub><mi mathvariant='italic'>F</mi>
<mn>0</mn>
</msub>
<mo stretchy="false">(</mo>
<mi mathvariant='italic'>u</mi>
<mo stretchy="false">)</mo>
<mo stretchy="false">+</mo>
<mo stretchy="false">⋯</mo>
<mo stretchy="false">+</mo>
<msub><mi mathvariant='italic'>w</mi>
<mrow><mi mathvariant='italic'>m</mi>
<mn>-1</mn>
</mrow>
</msub>
<mo stretchy="false">*</mo>
<msub><mi mathvariant='italic'>F</mi>
<mrow><mi mathvariant='italic'>m</mi>
<mn>-1</mn>
</mrow>
</msub>
<mo stretchy="false">(</mo>
<mi mathvariant='italic'>u</mi>
<mo stretchy="false">)</mo>
</mtd></mtr><mtr><mtd columnalign="right" >
<msub><mi mathvariant='italic'>W</mi>
<mn>1</mn>
</msub>
<mo stretchy="false">(</mo>
<mi mathvariant='italic'>u</mi>
<mo stretchy="false">)</mo>
</mtd><mtd columnalign="center" >
<mo stretchy="false">=</mo>
</mtd><mtd columnalign="left" >
<msub><mi mathvariant='italic'>w</mi>
<mn>0</mn>
</msub>
<mo stretchy="false">*</mo>
<msubsup><mi mathvariant='italic'>F</mi>
<mn>0</mn>
<mrow><mo stretchy="false">(</mo>
<mn>1</mn>
<mo stretchy="false">)</mo>
</mrow>
</msubsup>
<mo stretchy="false">(</mo>
<mi mathvariant='italic'>u</mi>
<mo stretchy="false">)</mo>
<mo stretchy="false">*</mo>
<msup><mi mathvariant='italic'>x</mi>
<mrow><mo stretchy="false">(</mo>
<mn>1</mn>
<mo stretchy="false">)</mo>
</mrow>
</msup>
<mo stretchy="false">+</mo>
<mo stretchy="false">⋯</mo>
<mo stretchy="false">+</mo>
<msub><mi mathvariant='italic'>w</mi>
<mrow><mi mathvariant='italic'>m</mi>
<mn>-1</mn>
</mrow>
</msub>
<mo stretchy="false">*</mo>
<msubsup><mi mathvariant='italic'>F</mi>
<mrow><mi mathvariant='italic'>m</mi>
<mn>-1</mn>
</mrow>
<mrow><mo stretchy="false">(</mo>
<mn>1</mn>
<mo stretchy="false">)</mo>
</mrow>
</msubsup>
<mo stretchy="false">(</mo>
<mi mathvariant='italic'>u</mi>
<mo stretchy="false">)</mo>
<mo stretchy="false">*</mo>
<msup><mi mathvariant='italic'>x</mi>
<mrow><mo stretchy="false">(</mo>
<mn>1</mn>
<mo stretchy="false">)</mo>
</mrow>
</msup>
</mtd></mtr></mtable>
</mrow></math>
This operation computes the derivatives
<math xmlns="http://www.w3.org/1998/Math/MathML" display="block"><mrow>
<mtable rowalign="center" ><mtr><mtd columnalign="right" >
<msubsup><mi mathvariant='italic'>W</mi>
<mn>0</mn>
<mrow><mo stretchy="false">(</mo>
<mn>1</mn>
<mo stretchy="false">)</mo>
</mrow>
</msubsup>
<mo stretchy="false">(</mo>
<mi mathvariant='italic'>u</mi>
<mo stretchy="false">)</mo>
</mtd><mtd columnalign="center" >
<mo stretchy="false">=</mo>
</mtd><mtd columnalign="left" >
<msub><mi mathvariant='italic'>w</mi>
<mn>0</mn>
</msub>
<mo stretchy="false">*</mo>
<msubsup><mi mathvariant='italic'>F</mi>
<mn>0</mn>
<mrow><mo stretchy="false">(</mo>
<mn>1</mn>
<mo stretchy="false">)</mo>
</mrow>
</msubsup>
<mo stretchy="false">(</mo>
<mi mathvariant='italic'>u</mi>
<mo stretchy="false">)</mo>
<mo stretchy="false">+</mo>
<mo stretchy="false">⋯</mo>
<mo stretchy="false">+</mo>
<msub><mi mathvariant='italic'>w</mi>
<mrow><mi mathvariant='italic'>m</mi>
<mn>-1</mn>
</mrow>
</msub>
<mo stretchy="false">*</mo>
<msubsup><mi mathvariant='italic'>F</mi>
<mrow><mi mathvariant='italic'>m</mi>
<mn>-1</mn>
</mrow>
<mrow><mo stretchy="false">(</mo>
<mn>1</mn>
<mo stretchy="false">)</mo>
</mrow>
</msubsup>
<mo stretchy="false">(</mo>
<mi mathvariant='italic'>u</mi>
<mo stretchy="false">)</mo>
</mtd></mtr><mtr><mtd columnalign="right" >
<msubsup><mi mathvariant='italic'>W</mi>
<mn>1</mn>
<mrow><mo stretchy="false">(</mo>
<mn>1</mn>
<mo stretchy="false">)</mo>
</mrow>
</msubsup>
<mo stretchy="false">(</mo>
<mi mathvariant='italic'>u</mi>
<mo stretchy="false">)</mo>
</mtd><mtd columnalign="center" >
<mo stretchy="false">=</mo>
</mtd><mtd columnalign="left" >
<msub><mi mathvariant='italic'>w</mi>
<mn>0</mn>
</msub>
<mo stretchy="false">*</mo>
<msup><mrow><mo stretchy="true">(</mo><mrow><msup><mi mathvariant='italic'>x</mi>
<mrow><mo stretchy="false">(</mo>
<mn>1</mn>
<mo stretchy="false">)</mo>
</mrow>
</msup>
</mrow><mo stretchy="true">)</mo></mrow>
<mrow><mstyle mathvariant='normal'><mi mathvariant='normal'>T</mi>
</mstyle></mrow>
</msup>
<mo stretchy="false">*</mo>
<msubsup><mi mathvariant='italic'>F</mi>
<mn>0</mn>
<mrow><mo stretchy="false">(</mo>
<mn>2</mn>
<mo stretchy="false">)</mo>
</mrow>
</msubsup>
<mo stretchy="false">(</mo>
<mi mathvariant='italic'>u</mi>
<mo stretchy="false">)</mo>
<mo stretchy="false">+</mo>
<mo stretchy="false">⋯</mo>
<mo stretchy="false">+</mo>
<msub><mi mathvariant='italic'>w</mi>
<mrow><mi mathvariant='italic'>m</mi>
<mn>-1</mn>
</mrow>
</msub>
<mo stretchy="false">*</mo>
<msup><mrow><mo stretchy="true">(</mo><mrow><msup><mi mathvariant='italic'>x</mi>
<mrow><mo stretchy="false">(</mo>
<mn>1</mn>
<mo stretchy="false">)</mo>
</mrow>
</msup>
</mrow><mo stretchy="true">)</mo></mrow>
<mrow><mstyle mathvariant='normal'><mi mathvariant='normal'>T</mi>
</mstyle></mrow>
</msup>
<msubsup><mi mathvariant='italic'>F</mi>
<mrow><mi mathvariant='italic'>m</mi>
<mn>-1</mn>
</mrow>
<mrow><mo stretchy="false">(</mo>
<mn>2</mn>
<mo stretchy="false">)</mo>
</mrow>
</msubsup>
<mo stretchy="false">(</mo>
<mi mathvariant='italic'>u</mi>
<mo stretchy="false">)</mo>
</mtd></mtr></mtable>
</mrow></math>
at
<math xmlns="http://www.w3.org/1998/Math/MathML" display="inline"><mrow>
<mi mathvariant='italic'>u</mi>
<mo stretchy="false">=</mo>
<msup><mi mathvariant='italic'>x</mi>
<mrow><mo stretchy="false">(</mo>
<mn>0</mn>
<mo stretchy="false">)</mo>
</mrow>
</msup>
</mrow></math>
.
<br/>
<br/>
<b><big><a name="f" id="f">f</a></big></b>
<br/>
The object
<code><i><font color="black"><span style='white-space: nowrap'>f</span></font></i></code>
has prototype
<code><font color="blue"><span style='white-space: nowrap'><br/>
     const ADFun<</span></font><i><font color="black"><span style='white-space: nowrap'>Base</span></font></i><font color="blue"><span style='white-space: nowrap'>> </span></font><i><font color="black"><span style='white-space: nowrap'>f</span></font></i><font color="blue"><span style='white-space: nowrap'><br/>
</span></font></code>
Before this call to <code><font color="blue">Reverse</font></code>, the value returned by
<code><font color="blue"><span style='white-space: nowrap'><br/>
     </span></font><i><font color="black"><span style='white-space: nowrap'>f</span></font></i><font color="blue"><span style='white-space: nowrap'>.size_order()<br/>
</span></font></code>
must be greater than or equal two (see <a href="size_order.xml" target="_top"><span style='white-space: nowrap'>size_order</span></a>
).
<br/>
<br/>
<b><big><a name="w" id="w">w</a></big></b>
<br/>
The argument
<code><i><font color="black"><span style='white-space: nowrap'>w</span></font></i></code>
has prototype
<code><font color="blue"><span style='white-space: nowrap'><br/>
     const </span></font><i><font color="black"><span style='white-space: nowrap'>Vector</span></font></i><font color="blue"><span style='white-space: nowrap'> &</span></font><i><font color="black"><span style='white-space: nowrap'>w</span></font></i><font color="blue"><span style='white-space: nowrap'><br/>
</span></font></code>
(see <a href="reverse_two.xml#Vector" target="_top"><span style='white-space: nowrap'>Vector</span></a>
below)
and its size
must be equal to
<code><i><font color="black"><span style='white-space: nowrap'>m</span></font></i></code>
, the dimension of the
<a href="seq_property.xml#Range" target="_top"><span style='white-space: nowrap'>range</span></a>
space for
<code><i><font color="black"><span style='white-space: nowrap'>f</span></font></i></code>
.
<br/>
<br/>
<b><big><a name="dw" id="dw">dw</a></big></b>
<br/>
The result
<code><i><font color="black"><span style='white-space: nowrap'>dw</span></font></i></code>
has prototype
<code><font color="blue"><span style='white-space: nowrap'><br/>
     </span></font><i><font color="black"><span style='white-space: nowrap'>Vector</span></font></i><font color="blue"><span style='white-space: nowrap'> </span></font><i><font color="black"><span style='white-space: nowrap'>dw</span></font></i><font color="blue"><span style='white-space: nowrap'><br/>
</span></font></code>
(see <a href="reverse_two.xml#Vector" target="_top"><span style='white-space: nowrap'>Vector</span></a>
below).
It contains both the derivative
<math xmlns="http://www.w3.org/1998/Math/MathML" display="inline"><mrow>
<msup><mi mathvariant='italic'>W</mi>
<mrow><mo stretchy="false">(</mo>
<mn>1</mn>
<mo stretchy="false">)</mo>
</mrow>
</msup>
<mo stretchy="false">(</mo>
<mi mathvariant='italic'>x</mi>
<mo stretchy="false">)</mo>
</mrow></math>
and the derivative
<math xmlns="http://www.w3.org/1998/Math/MathML" display="inline"><mrow>
<msup><mi mathvariant='italic'>U</mi>
<mrow><mo stretchy="false">(</mo>
<mn>1</mn>
<mo stretchy="false">)</mo>
</mrow>
</msup>
<mo stretchy="false">(</mo>
<mi mathvariant='italic'>x</mi>
<mo stretchy="false">)</mo>
</mrow></math>
.
The size of
<code><i><font color="black"><span style='white-space: nowrap'>dw</span></font></i></code>
is equal to
<math xmlns="http://www.w3.org/1998/Math/MathML" display="inline"><mrow>
<mi mathvariant='italic'>n</mi>
<mo stretchy="false">×</mo>
<mn>2</mn>
</mrow></math>
,
where
<math xmlns="http://www.w3.org/1998/Math/MathML" display="inline"><mrow>
<mi mathvariant='italic'>n</mi>
</mrow></math>
is the dimension of the
<a href="seq_property.xml#Domain" target="_top"><span style='white-space: nowrap'>domain</span></a>
space for
<code><i><font color="black"><span style='white-space: nowrap'>f</span></font></i></code>
.
<br/>
<br/>
<b><a name="dw.First Order Partials" id="dw.First Order Partials">First Order Partials</a></b>
<br/>
For
<math xmlns="http://www.w3.org/1998/Math/MathML" display="inline"><mrow>
<mi mathvariant='italic'>j</mi>
<mo stretchy="false">=</mo>
<mn>0</mn>
<mo stretchy="false">,</mo>
<mo stretchy="false">…</mo>
<mo stretchy="false">,</mo>
<mi mathvariant='italic'>n</mi>
<mo stretchy="false">-</mo>
<mn>1</mn>
</mrow></math>
,
<math xmlns="http://www.w3.org/1998/Math/MathML" display="block"><mrow>
<mi mathvariant='italic'>dw</mi>
<mo stretchy="false">[</mo>
<mi mathvariant='italic'>j</mi>
<mo stretchy="false">*</mo>
<mn>2</mn>
<mo stretchy="false">+</mo>
<mn>0</mn>
<mo stretchy="false">]</mo>
<mo stretchy="false">=</mo>
<mfrac><mrow><mo stretchy="false">∂</mo>
<msub><mi mathvariant='italic'>W</mi>
<mn>0</mn>
</msub>
</mrow>
<mrow><mo stretchy="false">∂</mo>
<msub><mi mathvariant='italic'>u</mi>
<mi mathvariant='italic'>j</mi>
</msub>
</mrow>
</mfrac>
<mrow><mo stretchy="true">(</mo><mrow><msup><mi mathvariant='italic'>x</mi>
<mrow><mo stretchy="false">(</mo>
<mn>0</mn>
<mo stretchy="false">)</mo>
</mrow>
</msup>
</mrow><mo stretchy="true">)</mo></mrow>
<mo stretchy="false">=</mo>
<msub><mi mathvariant='italic'>w</mi>
<mn>0</mn>
</msub>
<mo stretchy="false">*</mo>
<mfrac><mrow><mo stretchy="false">∂</mo>
<msub><mi mathvariant='italic'>F</mi>
<mn>0</mn>
</msub>
</mrow>
<mrow><mo stretchy="false">∂</mo>
<msub><mi mathvariant='italic'>u</mi>
<mi mathvariant='italic'>j</mi>
</msub>
</mrow>
</mfrac>
<mrow><mo stretchy="true">(</mo><mrow><msup><mi mathvariant='italic'>x</mi>
<mrow><mo stretchy="false">(</mo>
<mn>0</mn>
<mo stretchy="false">)</mo>
</mrow>
</msup>
</mrow><mo stretchy="true">)</mo></mrow>
<mo stretchy="false">+</mo>
<mo stretchy="false">⋯</mo>
<mo stretchy="false">+</mo>
<msub><mi mathvariant='italic'>w</mi>
<mrow><mi mathvariant='italic'>m</mi>
<mn>-1</mn>
</mrow>
</msub>
<mo stretchy="false">*</mo>
<mfrac><mrow><mo stretchy="false">∂</mo>
<msub><mi mathvariant='italic'>F</mi>
<mrow><mi mathvariant='italic'>m</mi>
<mn>-1</mn>
</mrow>
</msub>
</mrow>
<mrow><mo stretchy="false">∂</mo>
<msub><mi mathvariant='italic'>u</mi>
<mi mathvariant='italic'>j</mi>
</msub>
</mrow>
</mfrac>
<mrow><mo stretchy="true">(</mo><mrow><msup><mi mathvariant='italic'>x</mi>
<mrow><mo stretchy="false">(</mo>
<mn>0</mn>
<mo stretchy="false">)</mo>
</mrow>
</msup>
</mrow><mo stretchy="true">)</mo></mrow>
</mrow></math>
This part of
<code><i><font color="black"><span style='white-space: nowrap'>dw</span></font></i></code>
contains the same values as are returned
by <a href="reverse_one.xml" target="_top"><span style='white-space: nowrap'>reverse_one</span></a>
.
<br/>
<br/>
<b><a name="dw.Second Order Partials" id="dw.Second Order Partials">Second Order Partials</a></b>
<br/>
For
<math xmlns="http://www.w3.org/1998/Math/MathML" display="inline"><mrow>
<mi mathvariant='italic'>j</mi>
<mo stretchy="false">=</mo>
<mn>0</mn>
<mo stretchy="false">,</mo>
<mo stretchy="false">…</mo>
<mo stretchy="false">,</mo>
<mi mathvariant='italic'>n</mi>
<mo stretchy="false">-</mo>
<mn>1</mn>
</mrow></math>
,
<math xmlns="http://www.w3.org/1998/Math/MathML" display="block"><mrow>
<mi mathvariant='italic'>dw</mi>
<mo stretchy="false">[</mo>
<mi mathvariant='italic'>j</mi>
<mo stretchy="false">*</mo>
<mn>2</mn>
<mo stretchy="false">+</mo>
<mn>1</mn>
<mo stretchy="false">]</mo>
<mo stretchy="false">=</mo>
<mfrac><mrow><mo stretchy="false">∂</mo>
<msub><mi mathvariant='italic'>W</mi>
<mn>1</mn>
</msub>
</mrow>
<mrow><mo stretchy="false">∂</mo>
<msub><mi mathvariant='italic'>u</mi>
<mi mathvariant='italic'>j</mi>
</msub>
</mrow>
</mfrac>
<mrow><mo stretchy="true">(</mo><mrow><msup><mi mathvariant='italic'>x</mi>
<mrow><mo stretchy="false">(</mo>
<mn>0</mn>
<mo stretchy="false">)</mo>
</mrow>
</msup>
</mrow><mo stretchy="true">)</mo></mrow>
<mo stretchy="false">=</mo>
<munderover><mo displaystyle='true' largeop='true'>∑</mo>
<mrow><mo stretchy="false">ℓ</mo>
<mo stretchy="false">=</mo>
<mn>0</mn>
</mrow>
<mrow><mi mathvariant='italic'>n</mi>
<mn>-1</mn>
</mrow>
</munderover>
<msubsup><mi mathvariant='italic'>x</mi>
<mo stretchy="false">ℓ</mo>
<mrow><mo stretchy="false">(</mo>
<mn>1</mn>
<mo stretchy="false">)</mo>
</mrow>
</msubsup>
<mrow><mo stretchy="true">[</mo><mrow><msub><mi mathvariant='italic'>w</mi>
<mn>0</mn>
</msub>
<mo stretchy="false">*</mo>
<mfrac><mrow><msup><mo stretchy="false">∂</mo>
<mn>2</mn>
</msup>
<msub><mi mathvariant='italic'>F</mi>
<mn>0</mn>
</msub>
</mrow>
<mrow><mo stretchy="false">∂</mo>
<msub><mi mathvariant='italic'>u</mi>
<mo stretchy="false">ℓ</mo>
</msub>
<mo stretchy="false">∂</mo>
<msub><mi mathvariant='italic'>u</mi>
<mi mathvariant='italic'>j</mi>
</msub>
</mrow>
</mfrac>
<mrow><mo stretchy="true">(</mo><mrow><msup><mi mathvariant='italic'>x</mi>
<mrow><mo stretchy="false">(</mo>
<mn>0</mn>
<mo stretchy="false">)</mo>
</mrow>
</msup>
</mrow><mo stretchy="true">)</mo></mrow>
<mo stretchy="false">+</mo>
<mo stretchy="false">⋯</mo>
<mo stretchy="false">+</mo>
<msub><mi mathvariant='italic'>w</mi>
<mrow><mi mathvariant='italic'>m</mi>
<mn>-1</mn>
</mrow>
</msub>
<mo stretchy="false">*</mo>
<mfrac><mrow><msup><mo stretchy="false">∂</mo>
<mn>2</mn>
</msup>
<msub><mi mathvariant='italic'>F</mi>
<mrow><mi mathvariant='italic'>m</mi>
<mn>-1</mn>
</mrow>
</msub>
</mrow>
<mrow><mo stretchy="false">∂</mo>
<msub><mi mathvariant='italic'>u</mi>
<mo stretchy="false">ℓ</mo>
</msub>
<mo stretchy="false">∂</mo>
<msub><mi mathvariant='italic'>u</mi>
<mi mathvariant='italic'>j</mi>
</msub>
</mrow>
</mfrac>
<mrow><mo stretchy="true">(</mo><mrow><msup><mi mathvariant='italic'>x</mi>
<mrow><mo stretchy="false">(</mo>
<mn>0</mn>
<mo stretchy="false">)</mo>
</mrow>
</msup>
</mrow><mo stretchy="true">)</mo></mrow>
</mrow><mo stretchy="true">]</mo></mrow>
</mrow></math>
<br/>
<b><big><a name="Vector" id="Vector">Vector</a></big></b>
<br/>
The type
<code><i><font color="black"><span style='white-space: nowrap'>Vector</span></font></i></code>
must be a <a href="simplevector.xml" target="_top"><span style='white-space: nowrap'>SimpleVector</span></a>
class with
<a href="simplevector.xml#Elements of Specified Type" target="_top"><span style='white-space: nowrap'>elements of type</span></a>
<code><i><font color="black"><span style='white-space: nowrap'>Base</span></font></i></code>
.
The routine <a href="checksimplevector.xml" target="_top"><span style='white-space: nowrap'>CheckSimpleVector</span></a>
will generate an error message
if this is not the case.
<br/>
<br/>
<b><big><a name="Hessian Times Direction" id="Hessian Times Direction">Hessian Times Direction</a></big></b>
<br/>
Suppose that
<math xmlns="http://www.w3.org/1998/Math/MathML" display="inline"><mrow>
<mi mathvariant='italic'>w</mi>
</mrow></math>
is the <code><i>i</i></code>-th elementary vector.
It follows that for
<math xmlns="http://www.w3.org/1998/Math/MathML" display="inline"><mrow>
<mi mathvariant='italic'>j</mi>
<mo stretchy="false">=</mo>
<mn>0</mn>
<mo stretchy="false">,</mo>
<mo stretchy="false">…</mo>
<mo stretchy="false">,</mo>
<mi mathvariant='italic'>n</mi>
<mn>-1</mn>
</mrow></math>
<math xmlns="http://www.w3.org/1998/Math/MathML" display="block"><mrow>
<mtable rowalign="center" ><mtr><mtd columnalign="right" >
<mi mathvariant='italic'>dw</mi>
<mo stretchy="false">[</mo>
<mi mathvariant='italic'>j</mi>
<mo stretchy="false">*</mo>
<mn>2</mn>
<mo stretchy="false">+</mo>
<mn>1</mn>
<mo stretchy="false">]</mo>
</mtd><mtd columnalign="center" >
<mo stretchy="false">=</mo>
</mtd><mtd columnalign="left" >
<msub><mi mathvariant='italic'>w</mi>
<mi mathvariant='italic'>i</mi>
</msub>
<munderover><mo displaystyle='true' largeop='true'>∑</mo>
<mrow><mo stretchy="false">ℓ</mo>
<mo stretchy="false">=</mo>
<mn>0</mn>
</mrow>
<mrow><mi mathvariant='italic'>n</mi>
<mn>-1</mn>
</mrow>
</munderover>
<mfrac><mrow><msup><mo stretchy="false">∂</mo>
<mn>2</mn>
</msup>
<msub><mi mathvariant='italic'>F</mi>
<mi mathvariant='italic'>i</mi>
</msub>
</mrow>
<mrow><mo stretchy="false">∂</mo>
<msub><mi mathvariant='italic'>u</mi>
<mi mathvariant='italic'>j</mi>
</msub>
<mo stretchy="false">∂</mo>
<msub><mi mathvariant='italic'>u</mi>
<mo stretchy="false">ℓ</mo>
</msub>
</mrow>
</mfrac>
<mrow><mo stretchy="true">(</mo><mrow><msup><mi mathvariant='italic'>x</mi>
<mrow><mo stretchy="false">(</mo>
<mn>0</mn>
<mo stretchy="false">)</mo>
</mrow>
</msup>
</mrow><mo stretchy="true">)</mo></mrow>
<msubsup><mi mathvariant='italic'>x</mi>
<mo stretchy="false">ℓ</mo>
<mrow><mo stretchy="false">(</mo>
<mn>1</mn>
<mo stretchy="false">)</mo>
</mrow>
</msubsup>
</mtd></mtr><mtr><mtd columnalign="right" >
</mtd><mtd columnalign="center" >
<mo stretchy="false">=</mo>
</mtd><mtd columnalign="left" >
<msub><mrow><mo stretchy="true">[</mo><mrow><msubsup><mi mathvariant='italic'>F</mi>
<mi mathvariant='italic'>i</mi>
<mrow><mo stretchy="false">(</mo>
<mn>2</mn>
<mo stretchy="false">)</mo>
</mrow>
</msubsup>
<mrow><mo stretchy="true">(</mo><mrow><msup><mi mathvariant='italic'>x</mi>
<mrow><mo stretchy="false">(</mo>
<mn>0</mn>
<mo stretchy="false">)</mo>
</mrow>
</msup>
</mrow><mo stretchy="true">)</mo></mrow>
<mo stretchy="false">*</mo>
<msup><mi mathvariant='italic'>x</mi>
<mrow><mo stretchy="false">(</mo>
<mn>1</mn>
<mo stretchy="false">)</mo>
</mrow>
</msup>
</mrow><mo stretchy="true">]</mo></mrow>
<mi mathvariant='italic'>j</mi>
</msub>
</mtd></mtr></mtable>
</mrow></math>
Thus the vector
<math xmlns="http://www.w3.org/1998/Math/MathML" display="inline"><mrow>
<mo stretchy="false">(</mo>
<mi mathvariant='italic'>dw</mi>
<mo stretchy="false">[</mo>
<mn>1</mn>
<mo stretchy="false">]</mo>
<mo stretchy="false">,</mo>
<mi mathvariant='italic'>dw</mi>
<mo stretchy="false">[</mo>
<mn>3</mn>
<mo stretchy="false">]</mo>
<mo stretchy="false">,</mo>
<mo stretchy="false">…</mo>
<mo stretchy="false">,</mo>
<mi mathvariant='italic'>dw</mi>
<mo stretchy="false">[</mo>
<mi mathvariant='italic'>n</mi>
<mo stretchy="false">*</mo>
<mi mathvariant='italic'>q</mi>
<mo stretchy="false">-</mo>
<mn>1</mn>
<mo stretchy="false">]</mo>
<mo stretchy="false">)</mo>
</mrow></math>
is equal to the Hessian of
<math xmlns="http://www.w3.org/1998/Math/MathML" display="inline"><mrow>
<msub><mi mathvariant='italic'>F</mi>
<mi mathvariant='italic'>i</mi>
</msub>
<mo stretchy="false">(</mo>
<mi mathvariant='italic'>x</mi>
<mo stretchy="false">)</mo>
</mrow></math>
times the direction
<math xmlns="http://www.w3.org/1998/Math/MathML" display="inline"><mrow>
<msup><mi mathvariant='italic'>x</mi>
<mrow><mo stretchy="false">(</mo>
<mn>1</mn>
<mo stretchy="false">)</mo>
</mrow>
</msup>
</mrow></math>
.
In the special case where
<math xmlns="http://www.w3.org/1998/Math/MathML" display="inline"><mrow>
<msup><mi mathvariant='italic'>x</mi>
<mrow><mo stretchy="false">(</mo>
<mn>1</mn>
<mo stretchy="false">)</mo>
</mrow>
</msup>
</mrow></math>
is the <code><i>l</i></code>-th
<a href="glossary.xml#Elementary Vector" target="_top"><span style='white-space: nowrap'>elementary vector</span></a>
,
<math xmlns="http://www.w3.org/1998/Math/MathML" display="block"><mrow>
<mi mathvariant='italic'>dw</mi>
<mo stretchy="false">[</mo>
<mi mathvariant='italic'>j</mi>
<mo stretchy="false">*</mo>
<mn>2</mn>
<mo stretchy="false">+</mo>
<mn>1</mn>
<mo stretchy="false">]</mo>
<mo stretchy="false">=</mo>
<mfrac><mrow><msup><mo stretchy="false">∂</mo>
<mn>2</mn>
</msup>
<msub><mi mathvariant='italic'>F</mi>
<mi mathvariant='italic'>i</mi>
</msub>
</mrow>
<mrow><mo stretchy="false">∂</mo>
<msub><mi mathvariant='italic'>x</mi>
<mi mathvariant='italic'>j</mi>
</msub>
<mo stretchy="false">∂</mo>
<msub><mi mathvariant='italic'>x</mi>
<mo stretchy="false">ℓ</mo>
</msub>
</mrow>
</mfrac>
<mrow><mo stretchy="true">(</mo><mrow><msup><mi mathvariant='italic'>x</mi>
<mrow><mo stretchy="false">(</mo>
<mn>0</mn>
<mo stretchy="false">)</mo>
</mrow>
</msup>
</mrow><mo stretchy="true">)</mo></mrow>
</mrow></math>
<br/>
<b><big><a name="Example" id="Example">Example</a></big></b>
<br/>
The files
<a href="reverse_two.cpp.xml" target="_top"><span style='white-space: nowrap'>reverse_two.cpp</span></a>
and
<a href="hes_times_dir.cpp.xml" target="_top"><span style='white-space: nowrap'>hes_times_dir.cpp</span></a>
contain a examples and tests of reverse mode calculations.
They return true if they succeed and false otherwise.
<hr/>Input File: omh/reverse/reverse_two.omh
</body>
</html>