טכניון מכון טכנולוגי לישראל
הטכניון מכון טכנולוגי לישראל - בית הספר ללימודי מוסמכים  
M.Sc Thesis
M.Sc StudentFarida Nassar
SubjectIncorporating Metacognition into Usability Testing
DepartmentDepartment of Industrial Engineering and Management
Supervisors Professor Ackerman Rakefet
Full Professor Shtub Avraham
Full Thesis textFull thesis text - English Version


Abstract

Usability testing is an important phase in the development process of any software product, before launching it to the market. Usually, objective measures, like time to perform a task and success rates, are collected in usability tests, together with global subjective measures, such as satisfaction and ease of use. The present study applied the metacognitive approach to learning regulation to the domain of usability testing. I compared two user interfaces of a software tool designed to support project management learning. In addition to performing a complex project management simulation and self-reporting of global satisfaction, the participants performed a set of focused tasks. They answered a question regarding each task and rated their confidence in their answer. This confidence rating together with the commonly used objective measures of response time and accuracy allowed detailed examination of the associations between the three and generated a comprehensive set of measures. Global performance and satisfaction measures did not reveal differences between the two user interfaces. The detailed analysis proposed here exposed strengths and weaknesses in each of the user interfaces and provided a basis for practical recommendations for improving them. Importantly, better outcomes were found when reliable confidence was experienced. This finding suggests that a product that eliminates illusion of ease produces better outcomes. Overall, the study offers an applicable methodology for usability tests that takes into account metacognitive considerations for delving into the subjective experience of the users in more detail than done before.