|M.Sc Student||Zadorojniy Alexander|
|Subject||Constrained Markov Decision Processes with Application|
to Wireless Communications
|Department||Department of Electrical Engineering||Supervisor||Professor Adam Shwartz|
With the development of
personal communication services, portable terminals such as mobile telephones
and notebook computers are expected to be used more frequently and for longer
times. Hence power consumption will become even more important than it is now.
One of the major concerns in supporting such mobile applications is the energy
conservation and managements in mobile devices. In wireless communications, low
power may cause errors during the transmission. Therefore when we try to
conserve power we have to care about the Quality-of-Service.
This research deals with a derivation of new solution methods for constrained Markov decision processes and applications of these methods to the optimization of wireless communications. We intend to survey the existing methods of control, which involve control of power and delay, and investigate their effectiveness. We introduce a new type of power control in wireless communications, which minimizes the transmission delay under the average power constraint where at each slot uses one of the available discrete power levels, while maximal power level is limited by a peak power constraint. In this research we succeeded to derive two theorems that describe the structure of the optimal solution for general constrained Markov decision processes. Two additional theorems were derived for power saving problems. Moreover we developed innovative algorithm which is able to solve constrained Markov decision processes problems and investigate the sensitivity of the solution to changes in the constraint level.