On the basis of today's lecture you should be able to do Examples Sheet 3, #3, 4, 5.
I started the lecture with examples of two methods applied to solution of the continuous-time LQR problem.
Method 1 is the solution of this problem using the Hamilton Jacobi Bellman equation.
Method 2 foreshadows the use of Pontryagin's Maximum Principle, which we shall discuss in lectures 15, 16. You will need this for #6, 7, 8, 9, which should all be possible after lecture 15.
Question #10 I will probably do in lecture 16.
In the fishing example we were essentially using Theorem 14.1 to show that an optimal policy is one for which,
$$
u=\begin{cases}
0\\
a(\bar x)\\
u_{\max}
\end{cases}\text{as }\ x\begin{array}{c}
<\\
=\\
>
\end{array}\bar x
$$
and that a necessary condition is that $a'(\bar x)=\alpha$. At the end of page 59 of the notes, $F(x)$ is computed at a function of $T$, the time at which $x$ first becomes equal to $\bar x$. A simple calculation, which I did not do in lecture, is that $F_{xx}<0$, so $F$ is concave. All this information confirms that $F$ satisfies the HJB equation (14.2).
Notice that Figure 2 on page 59 is showing $u_{\max}\geq a(x)$ for all $x$, so fishing at maximum rate does cause the population to decrease (unless initially $a(x)=u_{\max}$).
I started the lecture with examples of two methods applied to solution of the continuous-time LQR problem.
Method 1 is the solution of this problem using the Hamilton Jacobi Bellman equation.
Method 2 foreshadows the use of Pontryagin's Maximum Principle, which we shall discuss in lectures 15, 16. You will need this for #6, 7, 8, 9, which should all be possible after lecture 15.
Question #10 I will probably do in lecture 16.
In the fishing example we were essentially using Theorem 14.1 to show that an optimal policy is one for which,
$$
u=\begin{cases}
0\\
a(\bar x)\\
u_{\max}
\end{cases}\text{as }\ x\begin{array}{c}
<\\
=\\
>
\end{array}\bar x
$$
and that a necessary condition is that $a'(\bar x)=\alpha$. At the end of page 59 of the notes, $F(x)$ is computed at a function of $T$, the time at which $x$ first becomes equal to $\bar x$. A simple calculation, which I did not do in lecture, is that $F_{xx}<0$, so $F$ is concave. All this information confirms that $F$ satisfies the HJB equation (14.2).
Notice that Figure 2 on page 59 is showing $u_{\max}\geq a(x)$ for all $x$, so fishing at maximum rate does cause the population to decrease (unless initially $a(x)=u_{\max}$).
