Stochastic Processes using Python

This unique introductory textbook provides a practical, pedagogical introduction to Monte Carlo Methods and Stochastic Processes. The book can be used for a wide variety of advanced undergraduate and first year graduate courses in Computational statistics, Introduction to stochastic processes, and Monte Carlo computational methods. The intended audience is advanced undergraduates and graduate students who take a stochastic processes course as well as researchers who need a practical introduction to Monte Carlo methods using Python.

The textbook contains 146 well documented Python examples integrated within the text, which demonstrate how to use Python for both numerical and symbolic calculations of stochastic processes. Each chapter concludes with a set of problems designed to help readers hone their skills in coding stochastic processes.

An introductory chapter introduces definitions and axioms of probability, conditional probability and Bayes' rule, with applications of finite stochastic processes. This is followed by Monte Carlo estimation of integrals and probabilities, and the rejection sampling and the inverse CDF transformation techniques.

Two chapters introduce simulations and sampling methods for all the classic continuous and discrete distributions, and example codes demonstrate applications of the central limit theorem. A detailed chapter is dedicated to variance reduction methods: stratified sampling, importance sampling, control and antithetic variables. Joint distributions are applied to Bayes' theorem, conditional expectation and variance, the laws of total expectation and total variance and convolution of random variables. The next chapter presents detailed simulations of the important Bernoulli and Poisson processes and their properties: distribution of events, arrival times, waiting times and autocorrelation, in the general context of stationary processes. The next chapter is dedicated to examples of random walks, gambler's ruin, the diffusion equation, and the solution of difference equations which leads to a discussion of Markov processes. Stochastic birth-death processes are presented next, with coded examples of the master equation, Kolmogorov equations and the Gillespie algorithm.

The book concludes with three detailed chapters on Markov chains, Monte Carlo (MCMC) methods and Bayesian Inference. Coded examples are given for transition matrices, stationary distributions, absorbing states, the Metropolis-Hastings algorithm, slice sampling, Gibbs sampling and Bayesian regression.

Key Features:

• A unique practical approach which merges presentation of the basic theory and the practical Python programming examples.
• Contains 139 well documented Python codes, which demonstrate clearly the theoretical and mathematical concepts, without overwhelming readers with mathematical details.
• All Python codes are available for downloading in the book's website at GitHub, in the form of ready to run Jupyter notebooks.
• Detailed PowerPoint presentations are available online, which are based on the material and examples from the textbook.
• A detailed solutions manual is also available for instructors using the textbook in their courses.