טכניון מכון טכנולוגי לישראל
הטכניון מכון טכנולוגי לישראל - בית הספר ללימודי מוסמכים  
M.Sc Thesis
M.Sc StudentItamar Einav
SubjectUsing Movie Subtitles for Creating Statistical Alignment
Models
DepartmentDepartment of Computer Science
Supervisor Professor Emeritus Alon Itai
Full Thesis textFull thesis text - English Version


Abstract

In this thesis we present a method for compiling a large-scale bilingual corpus from a database of movie subtitles. To create the corpus, we implemented an alignment algorithm which is based on Gale and Church’s sentence alignment algorithm (1993). Our algorithm not only relies on character length information, but also uses subtitle-duration information, which is encoded in the subtitle files. This results in a significant reduction in the alignment error rate. After creating a “subtitle-level” aligned corpus by finding the matching subtitle pairs, a statistical model is used for aligning the subtitles in the phrase level. The alignment is based on a joint probability model with second order dependence. Since the model is joint (as opposed to conditional) it is inherently symmetric, so no symmetrization step is required.

To find the best alignment for the corpus, as well as the best parameters of the model, we use an adapted version of the EM algorithm.