“School of Cognitive”

Back to Papers Home
Back to Papers of School of Cognitive

Paper   IPM / Cognitive / 11506
School of Cognitive Sciences
  Title:   A study on expertise of agents and its effects on cooperative Q-learning
  Author(s): 
1.  Babak Nadjar Arabi
2.  Sahar Mastoureshgh
3.  Majid Nili Ahmadabadi
  Status:   Published
  Journal: IEEE Transactions on Systems, Man and Cybernetics
  Vol.:  37
  Year:  2007
  Pages:   398-409
  Supported by:  IPM
  Abstract:
Cooperation in learning (CL) can be realized in a multiagent system, if agents are capable of learning from both their own experiments and other agents? knowledge and expertise. Extra resources are exploited into higher efficiency and faster learning in CL as compared to that of individual learning (IL). In the real world, however, implementation of CL is not a straightforward task, in part due to possible differences in area of expertise (AOE). In this paper, reinforcement-learning homogenous agents are considered in an environment with multiple goals or tasks. As a result, they become expert in different domains with different amounts of expertness. Each agent uses a one-step Q-learning algorithm and is capable of exchanging its Q-table with those of its teammates. Two crucial questions are addressed in this paper: ?How the AOE of an agent can be extracted?? and ?How agents can improve their performance in CL by knowing their AOEs?? An algorithm is developed to extract the AOE based on state transitions as a gold standard from a behavioral point of view. Moreover, it is discussed that the AOE can be implicitly obtained through agents? expertness in the state level. Three new methods for CL through the combination of Q-tables are developed and examined for overall performance after CL. The performances of developed methods are compared with that of IL, strategy sharing (SS), and weighted SS (WSS). Obtained results show the superior performance of AOE-based methods as compared to that of existing CL methods, which do not use the notion of AOE. These results are very encouraging in support of the idea that ?cooperation based on the AOE? performs better than the general CL methods.

Download TeX format
back to top
scroll left or right