M.Sc Thesis

M.Sc StudentLevy Yoav
SubjectAGrid: Semantic Model-Based Negotiating Agents as a Resource
Management Infrastructure in Computer Grids
DepartmentDepartment of Electrical and Computer Engineering
Supervisor ASSOCIATE PROFESSOR Yitzhak Birk


Computer Grid systems have evolved and proliferated in the last decade in the academy as well as in the industry. They enable a growing body of researchers to perform highly intensive computations in areas such as nuclear physics, gene research, the financial industry and more.

In a typical Grid system, the Resource Management Subsystem (RMS) is responsible for monitoring and controlling resource operation in the underlying computer nodes. WAN-induced latencies prevent the RMS to report on rapid fluctuations in resource availability in a timely manner, contributing to the inability of the matchmaker / scheduler to efficiently respond to such changes, downgrading their overall performance.

We present AGrid - an architecture for a two-tier resource allocation and job scheduling middleware, targeted at multi-cluster-based Grid systems. The contribution of this work is derived from the combination we have made of the following well-acknowledged approaches to system design:

(a)                      Interleaving of resource monitoring and allocation control loops between a global (and slow) tier and a cluster-local (and much faster) tier.

(b)                     Deployment and operation of specialized software agents within each compute  cluster, responsible for monitoring and (re-)allocating resources to jobs based on historical data

(c)                      Non-cooperative gaming theory that models the behavior of a group of participants with common goals but with different sensitivities to price and performance.

While the upper tier performs global (inter-cluster) matchmaking in a manner similar to contemporary middleware, in the lower tier local (intra-cluster) matchmaking / scheduling is performed, reacting to changes in resource availability. AGrid relies upon intra-cluster software agents to collect and disseminate key data regarding resource availability and job requirements, and performs adaptive matchmaking and scheduling, by means of negotiations among the user-representing agents. Local matchmaking enjoys order-of-magnitude lower latencies, thus enabling instant response. Economic models may be implemented in AGrid by means of algorithms that govern resource bidding, responding to changes in resource availability and job requirements. Bidding on desired resources can be performed by agents on behalf of users and resource owners based on economic models, exercising predefined user-level policies.

We further offer a robust software architecture that enables flexible and modular development of a distributed middleware implementation. The software architecture and supporting building blocks are detailed, and are demonstrated in this work by means of a simulative software prototype.