kilin> Software > Finite Markov Chain

KAGEYAMA Mai and FUKUDA Hiroshi,

(March 2008)

formula from John G.Kemeny and J.Laurie Snell,″Finite Markov Chains″(Springer-Verlag New York Heidelberg Berlin) A simpler version is here.

Input probability matrix P (P_{ij}, transition probability from i to j.): 1 0 0 0 0 0 0 1 0 0 0 0 1 0 0 1 0 0 0 0 1 0 1 0 0 0 0 1 0 1 0 1 0 0 1 0

state,level,equivalence class,cyclic class,cycle

A Chain With Transient Sets is one that has transient sets and ergodic sets You can calculate a Chain With Transient Sets by clicking a corresponding button label ″calculate″.

the mean of the total number of times N_{ij} the process is in a given transient state j from starting state i :N={N_{ij}}

the variance of N :N_{2}

the mean of the total number of steps τ_{i} needed to reach an ergodic set from state i :τ={τ_{2}}

the variance of τ : τ_{2}

the probability b_{ij}that the process starting in a transient state i ends up in an absorbing state j :B={b_{ij}}

the probability h_{ij} that the process will ever going to transient stte j,starting in transient state i :H={h_{ij}}

the mean number of transient states μ_{i} :μ={μ_{i}}

An Open Set is a set of transient states or a proper subset of an ergodic set. You can calculate an Open Set by clicking a corresponding button label ″calculate″. Input labels of states for an open set. Separate by a space.For example "1 3 4 ".

the mean number of times N_{ij} that the process is in state j before leaving an open set :N={N_{ij}}

the mean number of steps τ_{i} needed to leave an open set :τ={τ_{i}}

the variance of τ: τ_{2}

the probability Nρa_{i} that the process goes to state a where it leaves an open set :Nρa={Nρa_{i}}

A Regular Markov Chain is one that has no transient sets, and has a single ergodic set with only one cyclic class . You can calculate a Regular Markov Chain by clicking a corresponding button label ″calculate″.

limiting vector α_{i} : α={α_{i}}

the mean number of steps M_{ij} before entering state j for the first time after the initial position i : M={M_{ij}}

the variance of M : M_{2}

limiting covariance c_{ij} : C={c_{ij}}

S=A+A^{1}+A^{2}+.....A^{n}. A=Agency matrix.

Agency matrix between equivalence class.

distance(number of steps)

'th power of probability matrix