M.Sc Thesis | |

M.Sc Student | Korcia Gal Gila |
---|---|

Subject | Online Convex Optimization in the Random Order Model |

Department | Department of Electrical and Computers Engineering |

Supervisors | DR. Yehuda Levy |

DR. Dan Garber | |

Full Thesis text |

Online Convex
Optimization (OCO) is a powerful framework for sequential prediction,
portraying the natural uncertainty inherent in data-streams as though the data
were generated by an almost omniscient adversary. However, this view, which is
often too pessimistic for real-world data, comes with a price. The complexity
of solving many important online tasks in this adversarial framework becomes
much worse than that of their offline and even stochastic counterparts. In this
work we consider a natural random-order version of the OCO model, in which the
adversary can choose the set of loss functions but does not get to choose the
order in which they are supplied to the learner; Instead, they are observed in
uniformly random order. Focusing on two important families of online tasks, one
in which the cumulative loss function is strongly convex (though individual
loss functions may not even be convex), and the other being online *k*-PCA,
we show that under standard well-conditioned-data assumptions, standard online
gradient descent (OGD) methods become much more efficient in the random-order
model. In particular, for the first group of tasks OGD guarantees
poly-logarithmic regret. In the case of online *k*-PCA, OGD guarantees
sublinear regret using only a rank-*k* SVD on each iteration and memory
linear in the size of the solution.