Today we looked at some stopping problems. They crop up all over the place. They are present in financial mathematics in the theory of option pricing (Black-Scholes). The holder of an American option is allowed to exercise the right to buy (or sell) the underlying asset at a predetermined price at any time before or at the expiry date. The valuation of American options is essentially an optimal stopping problem.
You can read more about Bruss's odds algorithm in his paper: "Sum the odds to one and stop" Annals of Probability, Vol. 28, 1384–1391, (2000). I showed you how to use this to address a Secretary Problem in which the candidates arrive in groups, of sizes $n_1,n_2,\dotsc,n_h$. I wonder what is the best order to interview the groups? Should we see large groups first, or small groups? I forgot to mention that the optimal success probability when using the odds algorithm is $q_{s^*}\cdots q_n(r_{s^*}+\cdots+r_n)$ and that this is always at least $1/e$, provided $r_1+\cdots+r_n\geq 1$.
Let me finish with some hints about questions on Examples Sheets 1 and 2. Question 6 on Sheet 1 can be solved by realizing that there are only two stationary Markov policies. You simple need to decide which is best. Question 7 is a simple stopping problem that can be solved using a OSLA rule. Question 8 is tricky. You should use Theorem 4.2.
Question 9 on Sheet 1 is important because it should convince you that it is possible to solve stochastic dynamic programming equations using linear programming (at least if both state space and action space are finite.) It there are $n$ states and $m$ actions (controls) available in each state then we will have an LP in $n$ variables, with $nm$ linear inequality constraints.
When on sheet 2 you do Question 2 you might think about using Bruss's Odds Algorithm. Although you cannot apply the algorithm directly, you should be able to solve this problem using the same scheme of proof as we used to prove the optimality of the odds algorithm in this lecture.
You can read more about Bruss's odds algorithm in his paper: "Sum the odds to one and stop" Annals of Probability, Vol. 28, 1384–1391, (2000). I showed you how to use this to address a Secretary Problem in which the candidates arrive in groups, of sizes $n_1,n_2,\dotsc,n_h$. I wonder what is the best order to interview the groups? Should we see large groups first, or small groups? I forgot to mention that the optimal success probability when using the odds algorithm is $q_{s^*}\cdots q_n(r_{s^*}+\cdots+r_n)$ and that this is always at least $1/e$, provided $r_1+\cdots+r_n\geq 1$.
Let me finish with some hints about questions on Examples Sheets 1 and 2. Question 6 on Sheet 1 can be solved by realizing that there are only two stationary Markov policies. You simple need to decide which is best. Question 7 is a simple stopping problem that can be solved using a OSLA rule. Question 8 is tricky. You should use Theorem 4.2.
Question 9 on Sheet 1 is important because it should convince you that it is possible to solve stochastic dynamic programming equations using linear programming (at least if both state space and action space are finite.) It there are $n$ states and $m$ actions (controls) available in each state then we will have an LP in $n$ variables, with $nm$ linear inequality constraints.
When on sheet 2 you do Question 2 you might think about using Bruss's Odds Algorithm. Although you cannot apply the algorithm directly, you should be able to solve this problem using the same scheme of proof as we used to prove the optimality of the odds algorithm in this lecture.
