On the basis of today's lecture you should be able to do Examples Sheet 3, #3, 4, 5. Questions 7--10 use Pontryagin's maximum principle.
I started the lecture with examples of two methods applied to solution of the continuous-time LQR problem.
Method 1 is the solution of this problem using the Hamilton Jacobi Bellman equation.
Method 2 foreshadows the use of Pontryagin's Maximum Principle, which we shall discuss in lectures 15, 16. You will need this for #6, 7, 8, 9, 10, which should all be possible after lecture 15.
In the fishing example we were essentially using Theorem 14.1 to show that an optimal policy is one for which,
u={0a(ˉx)umaxas x<=>ˉx
and that a necessary condition is that a′(ˉx)=α. On page 60 of the notes, F(x) is computed at a function of T, the time at which x first becomes equal to ˉx. We also check that Fxx<0, so F is concave. All this information confirms that F satisfies the HJB equation (14.2).
Notice that Figure 2 on page 60 is showing umax≥a(x) for all x, so fishing at maximum rate does cause the population to decrease (unless initially a(x)=umax).
I started the lecture with examples of two methods applied to solution of the continuous-time LQR problem.
Method 1 is the solution of this problem using the Hamilton Jacobi Bellman equation.
Method 2 foreshadows the use of Pontryagin's Maximum Principle, which we shall discuss in lectures 15, 16. You will need this for #6, 7, 8, 9, 10, which should all be possible after lecture 15.
In the fishing example we were essentially using Theorem 14.1 to show that an optimal policy is one for which,
u={0a(ˉx)umaxas x<=>ˉx
and that a necessary condition is that a′(ˉx)=α. On page 60 of the notes, F(x) is computed at a function of T, the time at which x first becomes equal to ˉx. We also check that Fxx<0, so F is concave. All this information confirms that F satisfies the HJB equation (14.2).
Notice that Figure 2 on page 60 is showing umax≥a(x) for all x, so fishing at maximum rate does cause the population to decrease (unless initially a(x)=umax).
