M.Sc Student | Geiger Omer |
---|---|

Subject | Algorithmic Exam Generation |

Department | Department of Computer Science |

Supervisor | Professor Shaul Markovitch |

Full Thesis text |

Given a class of students, and a pool
of questions in the domain of study, what subset will constitute a “good”
exam? Millions of educators are dealing with this difficult problem worldwide,
yet the task of composing exams is still performed manually. In this work we
present a novel algorithmic framework for exam composition. Our main
formulation requires two input components: a student population represented by
a distribution over a set of overlay models, each consisting of a set of
mastered abilities, or actions; and a *target model ordering* that, given
any two student models, defines which should be graded higher. To determine the
performance of a student model on a potential question, we test whether it
satisfies a disjunctive action landmark, i.e., whether its abilities are
sufficient to follow at least one solution path. Based on these, we present a
novel utility function for evaluating exams. An exam is highly evaluated if it
is expected to order the student population with high correlation to the target
order. In an alternative formulation we devised, the target ordering is
replaced with a target grade mapping indicating the desired grade for each
student. In this case, good exams are those for which the expected grades are
close to those specified by the target mapping. The merit of our algorithmic
framework is exemplified with real auto-generated questions in two domains:
middle-school algebra and trigonometric equations.