kilin> Software > Finite Markov Chain

Calculator for Finite Markov Chain

KAGEYAMA Mai and FUKUDA Hiroshi,

(March 2008)

formula from John G.Kemeny and J.Laurie Snell,″Finite Markov Chains″(Springer-Verlag New York Heidelberg Berlin) A simpler version is here.

Input probability matrix P (Pij, transition probability from i to j.):


1,Classification of states and chains

state,level,equivalence class,cyclic class,cycle


2,Chains With Transient Sets

A Chain With Transient Sets is one that has transient sets and ergodic sets You can calculate a Chain With Transient Sets by clicking a corresponding button label ″calculate″.

the mean of the total number of times Nij the process is in a given transient state j from starting state i :N={Nij}

the variance of N :N2

the mean of the total number of steps τi needed to reach an ergodic set from state i :τ={τ2}

the variance of τ : τ2

the probability bijthat the process starting in a transient state i ends up in an absorbing state j :B={bij}

the probability hij that the process will ever going to transient stte j,starting in transient state i :H={hij}

the mean number of transient states μi :μ={μi}

Open Sets

An Open Set is a set of transient states or a proper subset of an ergodic set. You can calculate an Open Set by clicking a corresponding button label ″calculate″. Input labels of states for an open set. Separate by a space.For example "1 3 4 ".

the mean number of times Nij that the process is in state j before leaving an open set :N={Nij}

the variance of N :N2

the mean number of steps τi needed to leave an open set :τ={τi}

the variance of τ: τ2

the probability Nρai that the process goes to state a where it leaves an open set :Nρa={Nρai}


3,Regular Markov Chains

A Regular Markov Chain is one that has no transient sets, and has a single ergodic set with only one cyclic class . You can calculate a Regular Markov Chain by clicking a corresponding button label ″calculate″.

limiting vector αi : α={αi}

the mean number of steps Mij before entering state j for the first time after the initial position i : M={Mij}

the variance of M : M2

limiting covariance cij : C={cij}


4,Appendix

S=A+A1+A2+.....An. A=Agency matrix.

Agency matrix between equivalence class.

distance(number of steps)

'th power of probability matrix