At the end of Lecture 13 I introduced the Hamilton-Jacobi-Bellman equation, which is a continuous time version of the optimality equation. In Section 13.3 (which I did not discuss in lectures) there is a presentation of the HJB equation for the continuous-time version of our well-studied LQ regulation problem. This is essentially Examples Sheet 3, Question 3. So you can review that section when you do the examples sheet. It is straightforward.
I have just now added to the notes a Section 13.4. This section is not examinable, but I thought you might find it interesting. The HJB equation is used in this example to deduce how fisherman might either fish to extinction, or not, a potentially sustainable fish population. The population follows dynamics of $\dot x=a(x)-u$, when $x>0$, where $u$ is the rate at which fish are extracted. Fishermen are trying to maximize
$$\int_0^\infty u(t) e^{-\alpha t}dt.$$ Whether the fish population is completely wiped out, or sustained, depends on whether the discounting rate $\alpha$ is greater or less than the rate at which the fish population can grow when it is small, i.e. $a'(0)$. If $\alpha< a'(0)$ then the population will converge to a sustainable positive level $\bar x$ and at which the optimal fishing rate is $\bar u$ and $\dot x=a(\bar x)-\bar u=0$.
We are not going to spend much time thinking about how to solve HJB equations directly, because the theory of Pontryagin's Maximum Principle that we will meet in Lecture 14 is more powerful. However, Questions 4 and 5 are about find a solution to the HJB equation, and you begin these questions by writing down the infinitesimal form of the optimality equation.
I have just now added to the notes a Section 13.4. This section is not examinable, but I thought you might find it interesting. The HJB equation is used in this example to deduce how fisherman might either fish to extinction, or not, a potentially sustainable fish population. The population follows dynamics of $\dot x=a(x)-u$, when $x>0$, where $u$ is the rate at which fish are extracted. Fishermen are trying to maximize
$$\int_0^\infty u(t) e^{-\alpha t}dt.$$ Whether the fish population is completely wiped out, or sustained, depends on whether the discounting rate $\alpha$ is greater or less than the rate at which the fish population can grow when it is small, i.e. $a'(0)$. If $\alpha< a'(0)$ then the population will converge to a sustainable positive level $\bar x$ and at which the optimal fishing rate is $\bar u$ and $\dot x=a(\bar x)-\bar u=0$.
We are not going to spend much time thinking about how to solve HJB equations directly, because the theory of Pontryagin's Maximum Principle that we will meet in Lecture 14 is more powerful. However, Questions 4 and 5 are about find a solution to the HJB equation, and you begin these questions by writing down the infinitesimal form of the optimality equation.
