Recurrent and ergodic Markov chains

Today, we will look in more detail into convergence of Markov chains - what does it actually mean and how can we tell, given the transition matrix of a Markov chain on a finite state space, whether it actually converges. So suppose that we are given a Markov chain on a finite state space, with … Continue reading Recurrent and ergodic Markov chains

Monte Carlo methods and Markov chains – an introduction

In our short series on machine learning, we have already applied sampling methods several times. We have used and implemented Gibbs sampling, and so far we have simply accepted that the approach works. Time to look at this in a bit more detail in order to understand why it works and what the limitations of … Continue reading Monte Carlo methods and Markov chains – an introduction