Our purpose in writing this monograph is to give a comprehensive treatment of the subject. We define bandit problems and give the necessary foundations in Chapter 2. We give proofs unless they are very easy or the result is not used in the sequel. All results given have been incorporated into our style and notation.