The Gittins index theorem is one of the most beautiful results in the field of Markov decision processes. Its discovery and proof in 1974 is due to John Gittins. The proof I have given in today's lecture is very different to Gittins's original proof. It was first presented in On the Gittins index for multiarmed bandits. Ann. Appl. Prob. 2, 1024-33, 1992. Proof is pages 1026-27.
For today's lecture I presented some slides on the Gittins Index. You may enjoy looking through the entire talk. It will look better if you download the .pdf file to your computer and read the presentation with a .pdf viewer such as acrobat, rather than try to read within your browser.
The theory of the MABP and proof of the Gittins index theorem is non-examinable. However, I thought you would enjoy seeing this beautiful result and how it can be proved. Lectures 1-6 have covered everything you need to know in order to understand the Gittins index. Today's lecture has also been an opportunity to revise ideas that we already met in problems on job scheduling and pharmaceutical trials. On Examples sheet 2, #3 you are asked to calculate the Gittins indices in a simple example - which should help to test your understanding of today's lecture.
The prospecting problem in 6.5 and Weiztman's Pandora's boxes problem in 7.5 are really the same problem, and a simple example to which the Gittins index theorem provides the answer.
For today's lecture I presented some slides on the Gittins Index. You may enjoy looking through the entire talk. It will look better if you download the .pdf file to your computer and read the presentation with a .pdf viewer such as acrobat, rather than try to read within your browser.
The theory of the MABP and proof of the Gittins index theorem is non-examinable. However, I thought you would enjoy seeing this beautiful result and how it can be proved. Lectures 1-6 have covered everything you need to know in order to understand the Gittins index. Today's lecture has also been an opportunity to revise ideas that we already met in problems on job scheduling and pharmaceutical trials. On Examples sheet 2, #3 you are asked to calculate the Gittins indices in a simple example - which should help to test your understanding of today's lecture.
The prospecting problem in 6.5 and Weiztman's Pandora's boxes problem in 7.5 are really the same problem, and a simple example to which the Gittins index theorem provides the answer.
