Stochastic Control: Fundamentals and Applications

This textbook provides a structured introduction to stochastic control—the mathematical framework for decision-making in systems influenced by randomness. It explains how stochastic processes evolve, how decisions affect their trajectories, and which mathematical tools are required to model and analyze these interactions, finding rich applications in areas such as financial engineering, artificial intelligence, and industrial automation, to name a few.

The content progresses from probability preliminaries and Markov chains to dynamic programming, controlled Markov processes, asymptotic control problems, and ergodic and stability considerations. Additional appendices supply supporting material on matrices, convergence, analysis results, convexity, and Python-based simulations, enabling readers to review prerequisite concepts as needed.

Intended for upper-undergraduate and early graduate students, the book will benefit those in mathematics, engineering, actuarial science, and related disciplines. It is also a useful resource for instructors designing courses on probability, stochastic processes, or stochastic control. Readers with a background in calculus, linear algebra, and probability can use the text to develop the theoretical and computational skills required for the study and application of the methods contained within.