Computing the Expected Markov Reward Rates with St

Authors: Víctor Suñé

Publish Date: 2016/03/11

Volume: 19, Issue: 2, Pages: 445-485

PDF Link

Abstract

By combining in a novel way the randomization method with the stationary detection technique we develop two new algorithms for the computation of the expected reward rates of finite irreducible Markov reward models with control of the relative error The first algorithm computes the expected transient reward rate and the second one computes the expected averaged reward rate The algorithms are numerically stable Further it is argued that from the point of view of runtime computational cost for mediumsized and large Markov reward models we can expect the algorithms to be better than the only variant of the randomization method that allows to control the relative error and better than the approach that consists in employing iteratively the currently existing algorithms that use the randomization method with stationarity detection but allow to control the absolute error The performance of the new algorithms is illustrated by means of examples showing that the algorithms can be not only faster but also more efficient than the alternatives in terms of runtime computational cost in relation to accuracy

Keywords: