|M.Sc Student||Itamar Einav|
|Subject||Using Movie Subtitles for Creating Statistical Alignment|
|Department||Department of Computer Science||Supervisor||Professor Emeritus Alon Itai|
In this thesis we present a method for compiling a large-scale bilingual corpus from a database of movie subtitles. To create the corpus, we implemented an alignment algorithm which is based on Gale and Church’s sentence alignment algorithm (1993). Our algorithm not only relies on character length information, but also uses subtitle-duration information, which is encoded in the subtitle files. This results in a significant reduction in the alignment error rate. After creating a “subtitle-level” aligned corpus by finding the matching subtitle pairs, a statistical model is used for aligning the subtitles in the phrase level. The alignment is based on a joint probability model with second order dependence. Since the model is joint (as opposed to conditional) it is inherently symmetric, so no symmetrization step is required.
To find the best alignment for the corpus, as well as the best parameters of the model, we use an adapted version of the EM algorithm.