Markov Chain Monte Carlo (MCMC) simulation can be used to calculate sums
\begin{equation}\label{eq:20170427a}
I = \sum_a \pi_a f(a)
\end{equation}
One finds a Markov process $X_t$ with stationary distribution $\pi_a$, then the sum \eqref{eq:20170427a} is approximated by
\begin{equation*}
S =\frac{1}{N} \sum_{t=1}^N f(X_t)
\end{equation*}
One can prove that under certain assumptions,
\begin{equation*}
\lim_{N \to \infty} \frac{1}{N} \sum_{t=1}^N f(X_t) = \sum_a \pi_a f(a)
\end{equation*}
This is Birkhoff's ergodic theorem. In this post I illustrate the behaviour of $ES$ for large $N$.
