Bandit problems: Sequential Allocation of Experiments

Our purpose in writing this monograph is to give a comprehensive treatment of the subject. We define bandit problems and give the necessary foundations in Chapter 2. Many of the important results that have appeared in the literature are presented in later chapters; these are interspersed with new results. We give proofs unless they are very easy or the result is not used in the sequel. We have simplified a number of arguments so many of the proofs given tend to be conceptual rather than calculational. All results given have been incorporated into our style and notation. The exposition is aimed at a variety of types of readers. Bandit problems and the associated mathematical and technical issues are developed from first principles. Since we have tried to be comprehens­ ive the mathematical level is sometimes advanced; for example, we use measure-theoretic notions freely in Chapter 2. But the mathema­ tically uninitiated reader can easily sidestep such discussion when it occurs in Chapter 2 and elsewhere. We have tried to appeal to graduate students and professionals in engineering, biometry, econ­ omics, management science, and operations research, as well as those in mathematics and statistics. The monograph could serve as a reference for professionals or as a telA in a semester or year-long graduate level course.