M.Sc Thesis

M.Sc StudentTurchin Yulia
SubjectRule Tuning with Uncertain Events
DepartmentDepartment of Industrial Engineering and Management
Supervisors PROF. Avigdor Gal
DR. Segev Wasserkrug
Full Thesis textFull thesis text - English Version


There is a growing need for the use of active systems, systems that act automatically based on events. In many cases, providing such active functionality requires materializing the occurrence of relevant events. A widespread paradigm for enabling such materialization is Complex Event Processing (CEP), a rule based paradigm, which currently relies on domain experts to fully define the relevant rules.

These experts need to provide the set of basic events which serves as input to the rule, their inter-relationships, and the parameters of the events for determining whether a new event should be materialized. While it is reasonable to expect that domain experts will be able to provide some information regarding such rules, providing all the required details is a hard task, even for domain experts. Moreover, in many event driven applications, rule parameters, or even the rules themselves, may change over time, due to the dynamic nature of the domain. Such changes complicate even further the specification task, as the expert must constantly update the rule specification. Therefore, relying solely on expert input for understanding and updating the intricate relationships between events in the real-world and giving estimations regarding the impact each one has on the materialization of a new event, may cause rules to be specified incorrectly, and is therefore a significant impediment to applications of CEP. As a result, we seek additional support to the definition of rules, beyond expert opinion.

This work presents a mechanism for automating both the initial definition of rules and the update of rules over time. This mechanism combines partial information provided by the domain expert with machine learning techniques, and is aimed at improving the accuracy of event specification and materialization.  The proposed mechanism for rule specification consists of two main repetitive stages, namely rule prediction and rule correction.

We show that our general mechanism may be combined with different expert knowledge levels regarding the initial rule specification and for each level we provide various examples of rule set correction method implementation based on machine learning and metaheuristic techniques.  We also provide an integrative process of the general mechanism design. In this work, we include possible implementations for both stages, based on a statistical mechanism called Kalman Filter.

We test our algorithm using the MIT Lincoln Laboratory DARPA On-Line Intrusion Detection Evaluations data set.