Distributed social dilemma with competitive meta-players

Transcription

1 Intl. Trans. in Op. Res. 8 (21) 75±88 Distributed social dilemma with competitive meta-players Tomohisa Yamashita, Keiji Suzuki and Azuma Ohuchi Graduate School of Engineering, Hokkaido University, Kita 13, Nishi 8, Kita-ku, Sapporo, Hokkaido, 6, Japan tomohisa@complex.eng.hokudai.ac.jp Received 7 July 1999; received in revised form 26 April 2; accepted 3 May 2 Abstract In our research, we propose the solution to the `Distributed Social Dilemma' as an extended social dilemma model. In this model, in order to prevent the players from becoming freeloaders, we introduce competitive metaplayers. Our purpose is to evaluate, through agent-based simulation, how the meta-players effect to increase cooperation amongst players in a distributed social dilemma. In order to examine our proposed solution, we introduce the preference of the players to cooperate as the measure of the effect of the meta-players. On the basis of the result of the simulations, we show the degree of the effect of the competitive meta-players on the distribution social dilemma. Keywords: simulation, game theory, economic modeling 1. Introduction Social dilemmas are social con icts between public welfare and individual wants (Dawes, 1981; Hardin, 1969). This kind of con ict obstructs the development of society. The cause of social dilemmas is the limited amount of public goods and the sel sh behavior of individuals. We are faced with numerous social dilemmas every day, for example protecting the environment, conserving natural resources, and containing the world population (Thiagarajan, 1991). Generally, there are two major solutions to social dilemmas (Yamagishi and Hayashi, 1996). One is the structural change approach; this is a change of the social structure so that freeloaders cannot acquire high payoffs, for example through government, metering, and contracts. The other is the attitude change approach; this is the change of the attitude of each member so that each member voluntarily thinks that freeloading is not right, for example through morality, religion, and communication. Recently, distribution of computer network systems has developed rapidly. When autonomous agents are constantly on the move achieving their aims in distributed computer network systems, agents can consider one part of a distributed system as an arti cial society. In other words, agents can consider a distributed system as a set of multiple arti cial societies. Social dilemmas exist not only in the real world but also in arti cial societies. In an arti cial society, sel sh behavior by autonomous agents # 21 International Federation of Operational Research Societies. Published by Blackwell Publishers Ltd

2 76 T. Yamashita, K. Suzuki and A. Ohuchi / Intl. Trans. in Op. Res. 8 (21) 75±88 cause social dilemmas because each agent's behavior is based on its own free will and personal motive. Though there is a social dilemma situation in each arti cial society (object), each agent must select one arti cial society as a place of activity out of a number of arti cial societies. In previous studies on social dilemmas (Axelrod, 1986; Hegselmann, 1996), there is no extended game in order that the players can select one game out of a number of social dilemma games. General social dilemma games do not include a situation in which the players select game partners or the game to be played out of many players or games (McFadzean and Tesfatsion, 1997; Teiman et al., 1998). In our research, in order to represent such a situation, we propose that the players initially select one game from a number of social dilemma games and then decide whether to adopt a cooperation or defection stance in selecting a game. We de ne this game as a `Distributed Social Dilemma'. To give an example as distributed social dilemma, imagine a server±client system with servers on distributed computer network systems. In the system, each server processes the tasks requested by the clients, and the clients select one server to process the tasks. The individual purpose of the client is to make the servers nish its own tasks as quickly as possible. The client can determine which server it requests to process the tasks and whether it requests an ordinary quantity of the tasks (cooperation) or a large quantity (defection). There are social dilemmas for every server because each client makes decisions based on maximizing its own payoff, and the servers have a limited ability to process the tasks. If some clients request one certain server to process a large quantity of the tasks, and the amount of the tasks is beyond the processing ability of that server, the processing speed of the server is reduced. Both the clients requesting a large quantity, and other clients requesting an ordinary quantity, have to wait for the tasks to be nished because it takes the server a long time to process all the tasks. If the server has suf cient processing ability for the amount of the tasks, its processing speed is not reduced. Though a few clients request a large quantity of tasks at a given time, the processing speed of the server is not greatly reduced. It is more pro table for the clients to select the server with a few tasks than the server with many tasks. Therefore, in a distributed social dilemma, it is very important for the client to determine both whether to request an ordinary or a large quantity, and which server to request to process the tasks. In a distributed social dilemma, it is dif cult to develop con dence and togetherness using the attitude change approach, because each society consists of an unspeci ed quantity of the general public. Furthermore, it is more dif cult to design con dence and togetherness of autonomous agents in an arti cial society. On the basis of this dif culty, our purposes are to propose a solution based on the structural change approach to the distributed social dilemma, and to examine the effect of our proposed solution. In our research, in order to prevent the players from becoming freeloaders, we introduce `the metaplayer' as a new player into the distributed social dilemma. The meta-player imposes a penalty on defective players, and levies a tax on all players based on the purpose of maximization of its own payoff. If there is only one meta-player in this game, it may raise the tax and the penalty to his own advantage. To avoid this situation, one meta-player is assigned into each social dilemma game to generate competition among the meta-players. Competition among the meta-players may prevent the meta-players from behaving sel shly, because the players may not select games under the meta-players with high tax and penalty. We formulate distributed social dilemmas with meta-players using game theory. Furthermore, we consider distributed social dilemma as not a one-shot game, but an iterated game because social dilemmas generally depend on decision-making over a long period, rather than over a short period. In our previous research (Yamashita et al., 2), we con rmed that most players are not freeloaders

3 T. Yamashita, K. Suzuki and A. Ohuchi / Intl. Trans. in Op. Res. 8 (21) 75±88 77 in the simulation of distributed social dilemmas with the meta-players, that each player makes a decision based on the nite state machine (FSM) evolved by genetic algorithms (McFadzean and Tesfatsion, 1997; Tesfatsion, 1998). Contrary to these results, we con rmed that the players are often freeloaders in the simulation, that each player makes a decision based on the expected payoff (McFadzean and Tesfatsion, 1997; Tesfatsion, 1998). On the basis of the results of these simulations, the tendency of the players to cooperate depends on the decision-making mechanism. Here, our research focuses on the structure of the interaction among the players and the meta-players rather than the acquisition of the cooperative strategy by evolutionary methods. We can not evaluate the structure of the interaction, that is, our proposed solution consistently if the tendency of the player to cooperate depends on the decision-making mechanism. In order to evaluate the effects of our proposed solution, we need to control the tendency of the players to cooperate. Therefore, we introduce `the preference of the players to cooperation' (Uno and Namatame, 1998). As the preference to cooperation is greater, the players select cooperation more often, even if the players acquire lower payoffs by cooperation. By means of comparison with other cases, for example with meta-players and without meta-players, we con rm how competitive meta-players have the effect of increasing cooperative players. To compare and evaluate, we use agent-based simulation where each agent is regarded as the player or the meta-player (Axelrod, 1984; 1986; 1997; Epstein and Axtell, 1996; Hegselmann, 1996). 2. Distributed social dilemma with meta-players 2.1. Introduction of the meta-player In order to prevent the players from becoming freeloaders, we introduce `the meta-player' as a new player into distributed social dilemmas. The meta-player imposes a penalty on the defective players and levies a tax on all players. As a result, the payoff functions of the players are determined by not only the number of cooperative and defective players, but also the tax and the penalty. The payoff function of the meta-player is the amount of the tax and the penalty. The purpose of the meta-player is to maximize its own payoff by adjusting both the tax and the penalty Competition among the meta-players If there is only one meta-player in this game, it may raise the tax and the penalty sel shly. To avoid this monopolized situation, some kind of relation to restrain the sel sh behavior is required. In order to realize this restriction, meta-players are assigned into each social dilemma game to cause competition among the meta-players. Here, the strategy set of the players is also extended. The players can select not only cooperation or defection, but also the game as the player playing. The meta-players hence can not raise the tax and the penalty sel shly, because the meta-players obtain no payoff if no player selects the games assigned to sel sh meta-players. As a result, the meta-players are in a dilemma as to whether to raise or reduce the tax and the penalty. Therefore, the competition among meta-players may prevent the meta-players from behaving sel shly, even if the objective of each meta-player is maximizing its own payoff.

4 78 T. Yamashita, K. Suzuki and A. Ohuchi / Intl. Trans. in Op. Res. 8 (21) 75± Basic structures Basic structures of distributed social dilemmas with meta-players, i.e. the meta-player set, the strategy set, and the payoff function are as follows: Meta-player set M ˆf1,..., j,..., mg (1) Strategy set of meta-player j Ù Tax j, Ù Pen j Ù Tax j ˆftax 1,..., tax l g (2 < l) < tax k < tax max (tax max : maximum tax) (2) Ù Penj ˆfpen 1,..., pen l g (2 < l) < pen k < pen max ( pen max : maximum penalty) Payoff function of meta-player jg j (Tax j, Pen j ) g j (Tax j, Pen j ) ˆ (js j j js j j)tax j js j jpen j (3) Pen j ˆfpen k j1 < k < lg Tax j ˆftax k j1 < k < lg Player set N ˆf1,..., i,..., ng Strategy set of player i Ð i ˆfS j, S j jj ˆ 1,..., mg (4) S j : Cooperation in game j S j : Defection in game j Payoff function set of players F ˆff 1,..., f j,..., f m g (5) Payoff function f j of player i of game j with meta-player j js j j: the number of players selecting cooperation in game j (6) js j j: the number of the players selecting defection in game j In our research, we consider the situation that the penalty and the tax do not change the dilemma structure of the payoff function of the players directly, that is, the defective strategy always dominates Table 1 The payoff function f j of player i of game j with meta-player j B D < m js j j js j j js j j S j á m js jj js j j js j j D Tax j S j á m js ij js i j js i j D Tax j Pen j B D. m js j j js j j js j j á md Tax j Pen j á md Tax j Pen j

5 T. Yamashita, K. Suzuki and A. Ohuchi / Intl. Trans. in Op. Res. 8 (21) 75±88 79 the cooperative strategy, although the meta-players levy the tax and impose the penalty. Consequently, maximum penalty satis es the following inequality (7). < Pen max < B (7) In order to construct a simple model, assume that the payoff of the player is always more than the amount of penalty and tax, in the case that minimum payoff of the players is and maximum penalty satis es the inequality (7) and maximum tax satis es the following inequality (8). < Tax max < á md Pen max (8) 3. Game analysis with agent-based simulation 3.1. Purpose In a distributed social dilemma with meta-players, it is hard to analyze the behaviors of the players and meta-players, because the interactions among the players and the meta-players are complicated for game theory. Thus, in order to con rm how the players and meta-players behave in certain conditions, we conduct agent-based simulations where each agent is regarded as player or meta-player (Axelrod, 1984; 1986; 1997; Epstein and Axtell, 1996; Hegselmann, 1996) Decision-making mechanisms The players and the meta-players make decisions based on the expected payoffs for each strategy (McFadzean and Tesfatsion, 1997; Teiman et al., 1998). The expected payoffs of the players are in uenced by the preferences of the players to cooperation (Uno and Namatame, 1998) Expected payoff In this game, the players and the meta-players make decisions on the basis of continuously updated values, `the expected payoffs', that relate to the reactive decisions in the iterated game. A player or a meta-player has received a payoff from the game. It is recorded as history, and it has updated to re ect the receipt of the new payoff. The expected payoff is de ned as follows. Suppose that a player (or a meta-player) v selects the strategy s, the number that the player v selects the strategy s is T v,s, t-th payoff in a series of the payoff acquired by the strategy s, X v,s ˆ (x 1 v,s,..., xt v,s v,s ), is xv,s t and the discount factor is ä, the expected payoff of the player v for the strategy s is represented as follows. expected payoff v,s ˆ XT v,s ä t xv,s t : tˆ1 (9) Preference of player The preference of the player i for the strategy s is represented as p i,s, and in uences expected payoff i,s as follows.

6 8 T. Yamashita, K. Suzuki and A. Ohuchi / Intl. Trans. in Op. Res. 8 (21) 75±88 expected payoff i,s ˆ XT i,s ä t (x i,s p i,s ) t (1) tˆ1 In this research, the preference p i,s of the player i for the strategy s is represented by equation 11. p i,s ˆ p(i) if s 2 S j otherwise (11) The preference for each cooperative strategy is equal. The preference function p(i) is represented by Figure 1. The preference of the player i to cooperation is based on equation (12) below. The x axis is the player i and the y axis is the preference of the players to cooperation. The preference function p(i) in Figure 1 is arrived at by using equation (12), based on normal distribution (Uno et al., 1998). Equation (12) represents the relationship between the preference of the players to cooperation, p, and the number of players, n( p). " n( p) ˆ p A exp 1 # p ì 2 (12) 2ð 2 ó Strategy selection In this simulation, the rationality of the player and the meta-player is the selection of the maximum expected payoff. The player i selects the strategy s where expected payoff i,s is largest of all expected payoffs of the player i. s ˆ arg max s2ði X T i,s tˆ1 ä t (x i,s p i,s ) t (13) Similarly, the meta-player j also selects the strategy s where expected payoff j,s is largest in the expected payoffs of the meta-player j. s X T j,s ˆ arg max s2ù ä t x t j,s tˆ1 (14) the preference of the player to cooperation µ σ 2 µ µ σ 2 player i N Fig. 1. The preference function p(i).

7 4. Simulation T. Yamashita, K. Suzuki and A. Ohuchi / Intl. Trans. in Op. Res. 8 (21) 75± Conditions With regard to the relationship between the meta-players, there are two kinds of competition. One is the competition among the meta-players to maximize their own payoff. When there are either games with meta-players or without meta-players, the meta-player competes against the games without metaplayers. On this condition, as the preference of the players to cooperate is greater, the players select the cooperative strategy in the game without the meta-players more often because there is no tax. Therefore, the other is the competition against the games without meta-players. We observe how the numbers of cooperative and defective players change as the preference of the players to cooperate is greater on some conditions of the competition of the meta-players. In order to control the preference of the players to cooperate, we increase the coef cient ì of the preference function p(i) one by one from 4 to 18. According to the following parameter setting, if the preference of the player to cooperate is over 16, the player loses the incentive to freeload (defect). If the preference of the player to cooperation is under 4, the player loses the incentive to cooperate. In this paper, we conduct six kinds of simulations (cases 1±6). The conditions of the six simulations and the important parameters are shown in Tables 2 and 3. In Table 2, if meta-player is `on', there is a meta-player; if `off', there is no meta-player. If competition is `on', the purpose of the meta-players is maximization of their own payoff. If `off', the purpose of the meta-players is maximization of the amount of their payoff. For example, in case 3, there are two meta-players assigned to game 1 and game 2. There is no meta-player in game 3, that is, Tax 3 ˆ and Pen 3 ˆ. Each meta-player competes to maximize its own payoff Simulation results The results of the six simulations in cases 1±6 are shown in Figures 2 through 7. These gures are the average of three trials and represent the relations between the average number of the players and the coef cient ì of the preference function p(i). In these gures, the x axis is the average number of the players for the last 1 iterations, that is, from 29 to 3 iterations. The y axis is the coef cient ì of the preference function p(i). Table 2 The number of meta-players assigned to the game and the condition of the competition among the meta-players meta-player competition M game 1 game 2 game 3 case 1 off off off off case 2 1 off off on off case 3 2 off on on on case 4 2 off on on off case 5 3 on on on on case 6 3 on on on off

8 82 T. Yamashita, K. Suzuki and A. Ohuchi / Intl. Trans. in Op. Res. 8 (21) 75±88 Table 3 The common parameters in the simulations of cases 1±6 number of players jnj ˆ1 number of iterations T ˆ 25 maximum penalty Pen max ˆ 16 maximum tax Tax max ˆ 4 memory weight ä ˆ :9 range of coef cient 4 < ì < 18 strategy set of tax Ù Tax j ˆf, 5, 1, 15, 2, 25, 3, 35, 4g strategy set of penalty Ù Penj ˆf, 16, 32, 48, 64, 8, 96, 112, 128, 144, 16g coef cient of payoff function B ˆ 16 coef cient of payoff function D ˆ 2 coef cient of payoff function á ˆ 4 coef cient of normal distribution A ˆ 4:54 coef cient of normal distribution ó ˆ 1:5 Figure 2 shows the results of case 1, where there were no meta-players. Throughout this simulation, the number of players selecting the payoff function 1±3 were equal. Until the coef cient ì ˆ, there were no cooperative players. From the coef cient ì ˆ, cooperative players began to increase rapidly. From the coef cient ì ˆ 8, cooperative players increased gently. After the coef cient ì ˆ 35, there were no defective players. Figure 3 shows the results of case 2. There was one meta-player and it was assigned to the payoff function f 1. Throughout this simulation, there were no players selecting the defective strategy S 1. Until the coef cient ì ˆ, there were no cooperative players and all players were freeloaders in game 2 and 3. From the coef cient ì ˆ, the players selecting the cooperative strategy S 1 increased rapidly with decrease of players selecting the defective strategies S 2 and S 3. From the coef cient ì ˆ 9, the players selecting the cooperative strategies S 2 and S 3 increased rapidly with decreasing players selecting the cooperative strategy S 1. Figure 4 shows the result of case 3. There were two meta-players and each was assigned to the payoff functions f 1 and f 2. Two meta-players were competitive to maximize their own payoff. Throughout this 1 number of players S 1 S 2 S 3 S 1 S 2 S Fig. 2. The average number of players per coef cient ì of the preference function p(i) in case 1.

9 T. Yamashita, K. Suzuki and A. Ohuchi / Intl. Trans. in Op. Res. 8 (21) 75±88 83 number of players S 1, S 2, S 3 S 3 S 2 S Fig. 3. The average number of players per coef cient ì of the preference function p(i) in case 2. 1 S 3 S 2 S 1 number of players S Fig. 4. The average number of players per coef cient ì of the preference function p(i) in case 3. simulation, there were no players selecting the defective strategies S 1 and S 2. Until the coef cient ì ˆ, there were no cooperative players. From the coef cient ì ˆ, the players selecting the cooperative strategies S 1 and S 2 increased and there was a decrease in the selection of the defective strategy S 3. The players selecting the cooperative strategy S 1 ±S 3 increased rapidly. From the coef cient ì ˆ 16, most players selected the cooperative strategies S 1. Figure 5 shows the result of case 4. There were two meta-players and each was assigned to the payoff functions f 1 and f 2. Two meta-players were not competitive because their purpose was to maximize the amount of their payoff. Throughout this simulation, the transition of the number of cooperative and defective players was similar to that in the simulation of case 3. From the coef cient ì ˆ 13, most players selected the cooperative strategies S 1. Figure 6 shows the result of case 5. There were three meta-players and each meta-player was assigned to the payoff functions f 1 ± f 3. Three meta-players were competitive to maximize their own payoffs. Until the coef cient ì ˆ, there were no cooperative players. From the coef cient ì ˆ, the

10 84 T. Yamashita, K. Suzuki and A. Ohuchi / Intl. Trans. in Op. Res. 8 (21) 75±88 1 S 3 S 2 S 1 number of players S Fig. 5. The average number of players per coef cient ì of the preference function p(i) in case 4. number of players S 1, S 2, S 3 S 1, S 2, S Fig. 6. The average number of players per coef cient ì of the preference function p(i) in case 5. players selecting the cooperative strategies S 1 ±S 3 increased, with decreasing defective strategies S 1 ±S 3. After the coef cient ì ˆ 8, there were no freeloaders. Figure 7 shows the result of case 6. There were three meta-players and each meta-player was assigned to the payoff functions f 1 ± f 3. Three meta-players were not competitive because their purpose was to maximize the amount of their payoff. Throughout this simulation, the transition of the number of cooperative and defective player was similar to that in the simulation of case Discussion 5.1. Effect of meta-players In order to compare the cases with and without meta-players and the cases with and without competition among meta-players, we de ne `the effect of the meta-players'.

11 T. Yamashita, K. Suzuki and A. Ohuchi / Intl. Trans. in Op. Res. 8 (21) 75±88 85 number of players The effect of the meta-players is de ned as the increment of cooperative players by the meta-players per unit payoff of the meta-players charging from the players. In order to represent the effect of the meta-players, suppose that js 1 j,t j is the number of players selecting the cooperative strategy S j, at the t- th game in case 1. Similarly, js C j,t j is the number of players selecting the cooperative strategy S j, at the t-th game in case C (C ˆ 2,..., 6) in Figure 8. Then x C j,t is the payoff of the meta-player j at the t-th game in case C based on Figure 9. As a result, the effect of the meta-players in case C is represented by the following equation. 8 9 X T < X jfj = js C j,t j XjFj js 1 j,t : j ; tˆ j2f j2f effect C ˆ (15) X T X jfj x C j,t tˆ j2f S 1, S 2, S 3 S 1, S 2, S 3 Fig. 7. The average number of players per coef cient ì of the preference function p(i) in case 6. number of cooperative players case 6 case 5 case 1 case 2 case 4 case Fig. 8. The number of cooperative player in cases 1±6.

12 86 T. Yamashita, K. Suzuki and A. Ohuchi / Intl. Trans. in Op. Res. 8 (21) 75±88 16 payoff of meta-players case 6 case 5 case case 2 case 3 Fig. 9. The amount payoff of the meta-players in cases 2±6. The results of the calculation for cases 2±6 are represented in Figure Comparison of effect On the basis of Figure 1, the transition of the effect of the meta-players is classi ed into four characteristic cases. The rst case is that the coef cient ì satis es 8 < ì <, that is, as the preference of the players to cooperation increases a little without the meta-players, the number of cooperative players increases a lot. In this case, the players are sensitive to the difference between the payoff for cooperation and defection because the preference of the players to cooperation is not very.1 case 3 case 2.5 effect of meta-players.5.1 case 5 case 6 case Fig. 1. The effects of the meta-players of cases 2±6.

13 T. Yamashita, K. Suzuki and A. Ohuchi / Intl. Trans. in Op. Res. 8 (21) 75±88 87 great. The meta-players must adjust the tax and the penalty to such sensitive players. Then, the competition of the meta-players works to decrease the tax and the penalty. Though the number of cooperative players is not much different in each case, the amounts of the payoff of the meta-player in cases 5 and 6 are more than that in cases 2, 3, and 4. Therefore, in this case, the competition against the game without the meta-player is useful to increase the effect of the meta-players. The second case is where the coef cient ì satis es < ì < 7, that is, as the preference of the players to cooperation increases a lot without the meta-players, the number of cooperative players increases a little. The effects of the meta-players in cases 2±6 are similar, though the type of the competition of the meta-players is different in each case. Therefore, in this case, the effect of the metaplayers is not in uenced much by the difference of the competition of the meta-players. The third case is that the coef cient ì satis es 7 < ì < 14, that is, most players select cooperation without the meta-players. Though the number of cooperative players is not much different in each case, the amounts of the payoff of the meta-player in cases 3±6 are more than in case 2. In this case, the amount of the tax and the penalty charging for the player decreases as the number of metaplayers decreases. Therefore, the effect of the meta-players is in uenced not by the competition of the meta-players but the number of meta-players. The nal case is that the coef cient ì satis es 14 < ì < 18, that is, all players select cooperation without the meta-players. According to the parameter setting, if the preference of the player to cooperation is over 16, the player loses the incentive to freeload (defect). Therefore, if the coef cient ì is near 16, most players select cooperation. In this situation the players select the game without the meta-players because there is no tax. According to the simulation condition, because the preference to cooperation is based on the normal distribution, the player with a low preference for cooperation still exists. Furthermore, as the condition of cases 3 and 4, there are two meta-players and there is one game without the meta-player. The players with a high preference to cooperation select the game without the meta-player because there is no tax. Similarly, the players with low preference to cooperation select the game without the meta-player because there is no tax and no penalty. As a result, they always select defection because they can always freeload cooperative players and acquire a high payoff. If there are some games without meta-players, cooperative players can avoid freeloaders by moving to other games without meta-players. Consequently, the situation that there is only one game without a meta-player causes the negative effect of the meta-players. In this case, because the effect of case 3 is more than that of case 4, the competition among meta-players is useful to increase the effect of the meta-players. Furthermore, throughout the simulation, the effect of case 5 is more than that of case 6. Therefore, the competition among the meta-players increases the effect of the meta-player. 6. Conclusion In our research, we proposed the game `Distributed Social Dilemma', that the players select one game from a choice of social dilemma games. In order to prevent the players from becoming freeloaders, we introduced the meta-player into the distributed social dilemma. To evaluate the effect of competitive meta-players on increasing cooperative players, we introduce the preference of the players to cooperation as the measure of effect of the meta-players. By comparing cases through agent-based simulation, we con rm the degree of the effect of the competitive meta-players.

14 88 T. Yamashita, K. Suzuki and A. Ohuchi / Intl. Trans. in Op. Res. 8 (21) 75±88 References Axelrod, R The evolution of cooperation. Basic Books, New York. Axelrod, R An Evolutionary Approach to Norms. American Political Science Review, 8, 195±1111. Axelrod, R The Complexity of Cooperation. Princeton University Press, Princeton. Dawes, R.M Social Dilemmas. Annual Review of Psychology, 31, 169±193. Epstein, J.M., Axtell, R Growing Arti cial Societies. The MIT Press, Washington DC. Hardin, G The tragedy of the commons. Science, 162, 1243±1248. Hegselmann, R Social Dilemma in Lineland and Flatland, Frontiers in Social Dilemmas Research, 337±362. McFadzean, D., Tesfatsion, L A C Platform for the Evolution of Trade Networks. Working Paper. Department of Economics, Iowa State University. Tesfatsion, L Gale-Shapley Matching in an Evolutionary Trade Network Game. ISU Economic Report 43. Department of Economics, Iowa State University. Thiagarajan, S Garbage: A card game that simulates the trade-off between competition and concern Simulation & Games, 22, 112±115. Teiman, A.F., Houba, H., van der Laan, G Cooperation in a Multi-Dimension Local Interaction Model. Working paper. WoTinbergen Institute and Free University. Uno, K., Namatame, A An Evolutionary Design of Commitment Networks. Proceedings of The Fourth International Symposium on Arti cial Life and Robotics, 2, 528±531. Yamagishi, T. Hayashi, N Selective Play: Social Embeddedness of Social Dilemmas, Frontiers in Social Dilemmas Research, 363±384. Springer, New York. Yamashita, T., Suzuki, K., Ohuchi, A. 2. The Consideration of Behavior Process of Players in Social Dilemma of Iterated Multiple Lake Game, Journal of the Society of Instrument and Control Engineers, 36, 2, 195±23.