This book discusses both the theory and applications of Markov chains. The author studies both discrete-time and continuous-time chains and connected topics such as finite Gibbs fields, non-homogeneous Markov chains, discrete time regenerative processes, Monte Carlo simulation, simulated annealing, and queueing networks are also developed in this accessible and self-contained text. The text is firstly an introduction to the theory of stochastic processes at the undergraduate