M.Sc Thesis

M.Sc StudentZadorojniy Alexander
SubjectConstrained Markov Decision Processes with Application
to Wireless Communications
DepartmentDepartment of Electrical and Computer Engineering
Supervisor PROFESSOR EMERITUS Adam Shwartz


With the development of personal communication services, portable terminals such as mobile telephones and notebook computers are expected to be used more frequently and for longer times. Hence power consumption will become even more important than it is now. One of the major concerns in supporting such mobile applications is the energy conservation and managements in mobile devices. In wireless communications, low power may cause errors during the transmission. Therefore when we try to conserve power we have to care about the Quality-of-Service.

This research deals with a derivation of new solution methods for constrained Markov decision processes and applications of these methods to the optimization of wireless communications. We intend to survey the existing methods of control, which involve control of power and delay, and investigate their effectiveness. We introduce a new type of power control in wireless communications, which minimizes the transmission delay under the average power constraint where at each slot uses one of the available discrete power levels, while maximal power level is limited by a peak power constraint. In this research we succeeded to derive two theorems that describe the structure of the optimal solution for general constrained Markov decision processes. Two additional theorems were derived for power saving problems. Moreover we developed innovative algorithm which is able to solve constrained Markov decision processes problems and investigate the sensitivity of the solution to changes in the constraint level.