arxiv:adap-org/ v2 3 Sep 1997

Transcription

1 Emergence of Cooperaion and Organizaion in an Evoluionary Game D. Challe and Y.-C. Zhang Insiu de Physique Théorique, Universié de Fribourg, 1700 Fribourg, Swizerland arxiv:adap-org/ v2 3 Sep 1997 binary game is inroduced and analysed. N players have o choose one of he wo sides independenly and hose on he minoriy side win. Players uses a finie se of ad hoc sraegies o make heir decision, based on he pas record. The analysing power is limied and can adap when necessary. Ineresing cooperaion and compeiion paern of he sociey seem o arise and o be responsive o he payoff funcion. Mos curren economics heories are deducive in origin. One assumes ha each paricipan knows wha is bes for him given ha all oher paricipans are equally inelligen in choosing heir bes acions. However, i is recenly realised ha in he real world he acual players do no have he perfec foresigh and hindsigh, mos ofen heir acions are based on rial and error inducive hinking, raher han he deducive raionale assuming ha here are underlying firs principles. Wheher deducive or inducive hinking is more relevan is sill under debae [1]. Evoluionary games have also been sudied wihin he sandard framework of game heory [2]. However i is recenly poined ou ha he approach radiionally used in economics is no convenien o generalise o include irraionaliy, and an alernaive langevin ype equaion is proposed [3]. s physiciss we would like o view a game wih a large number of players a saisical sysem, we need o explore new approaches in which emerging collecive phenomena can be beer appreciaed. One recen approach using bounded raionaliy is paricularly inspiring, pu forward by B. rhur in his El Farol bar problem [4]. Following he similar philosophy, in his work we propose and sudy a simple evoluionary game. Le us consider a populaion of N (odd) players, each has some finie number of sraegies S. each ime sep, everybody has o choose o be in side or side B. The payoff of he game is o declare ha afer everybody has chosen side independenly, hose who are in he minoriy side win. In he simplies version, all winners collec a poin. The players make decisions based on he common knowledge of he pas record. We furher limi he record o conain only yes and no e.g. he side is he winning side or no, wihou he acual aendance number. Thus he sysem s signal can be represened by a binary sequence, meaning is he winning side (1) or no (0). Le us assume ha our players are quie limied in heir analysing power, hey can only reain las M bis of he sysem s signal and make heir nex decision basing only on hese M bis. Each player has a finie se of sraegies. sraegy is defined o be he nex acion (o be in or B) given a specific signal s M bis. The example of one sraegy is illusraed in able 1 for M=3. signal predicion There are 8 (= 2 M ) bis we can assign o he righ side, each configuraion corresponds a disinc sraegy, his makes he oal number of sraegy o be 2 2M = 256. This is indeed a fas increasing number, for M = 2, 3, 4, 5 i is 16, 256, 65536, We randomly draw S sraegies for each player, and some sraegies maybe by chance shared. However for moderaely large M, he chance of repeiion of a single sraegy is exceedingly small. noher special case is o have all 1 s (or 0 s) on he RHS of he able, corresponding o he fixed sraegy of saying a one side no maer wha happens. 1

2 Le us analyse he srucure of his Minoriy s game o see wha o expec. Consider he exreme case where only one player akes a side, all he ohers ake he oher. The lucky player ges a reward poin, nohing for he ohers. Equally exreme example is ha when (N 1)/2 players in one side, (N + 1)/2 on he oher. From he sociey poin of view, he second siuaion is preferable since he whole populaion ges (N 1)/2 poins whereas in he firs example only one poin a huge wase. Perfec coordinaion and iming would approach he 2nd, disaser would be he firs example. In general we expec he populaion o behave beween he above wo exremes. This binary game can be easily simulaed for a large populaion of players. Iniially, each player draws randomly one ou of his S sraegies and use i o predic nex sep, an arificial signal of M bis is also given. ll he S sraegies in a player s bag can collec poins depending if hey would win or no given he M pas bis, and he acual oucome of he nex play. However, hese poins are only virual poins as hey record he meri of a sraegy as if i were used each ime. The player uses he sraegy having he highes accumulaed poins (capial) for his acion, he ges a real poin only if he sraegy used happens o win in he nex play (a) FIG. 1. cual number of aendance a he side agains ime, for a populaion of 1001 players, having brain size of 6, 8 and 10 bis (a),(b),(c) In Figs. 1 we plo he acual number of aendance a he side, for a populaion of 1001 players, having various brain sizes (i.e. M bis). s one may expec, ha he emporal signal indeed flucuaes around he 50 %. Whoever akes he side wins a poin a a given ime sep when he signal is below 501. The precise number is no known o he players, hey only know if a side is winning or no, afer heir be is made. Noe ha large flucuaions imply large wase since sill more players could have aken he winning side wihou harm done o he ohers. On he oher hand, smaller flucuaions imply more efficien usage of available resources, in general his would require coordinaion and cooperaion which are no buil-in explicily. We see ha he populaion having larger brains (i.e. M larger) cope wih each oher beer : he flucuaion are indeed in decreasing order for ever increasingly inelligen players (i.e. M = 6, 8, 10). Remarkable is ha each player is by definiion selfish, no considerae o fellow (b) (c) 2

3 players, ye somehow hey manage o beer somewha share he limied available resources. Le us remark ha he very simples sraegy by playing randomly is no included here, for generaing random numbers more bis are needed. In a perfec iming, he average gain in he populaion would be 1/2 per play. Wase is proporional o flucuaion s ampliude hence he average gain is always below 1/2 in realiy. Since he game is symmerical in and B, one may be emped o use he simple sraegy o say a or B, hoping o ge exacly 1/2 gain. Le us menion if his sraegy indeed rewards 1/2 gain on average, many would imiae. Le us say here is a group siing a no maer wha signal is shown (his is included in he sraegy space). The acive players will soon recognise ha hey win less ofen choosing han B. In fac, for hem he game is no longer symmerical and hey will adop accordingly so ha he apparen advanage disappears for hose siing a one side fixed. This is similar o he arbirage opporuniies in finance: any obvious advanage will be arbiraged away no easy risk-free way o make a living boh for our players and hose in he real world. force unequally equiped players o play ogeher. One may fear ha he poorly brained players may ge exploied by he more powerfully brained ones. Indeed his is he case. We plo he average gain per ime sep afer a long ime. We see ha wihin a sub-populaion (same M) here are beer and worse performers. We have noiced ha beer players do no necessarily say ha way for a long ime, bu excepions exis. For M = 1, here appears fewer poins, since here are more degeneracies. s a group he more inelligen players gain more and he spread beween he rich and he poor is smaller, even hough he in-fighing among hem is more inensified. Remark ha above a cerain size (M 6) he average performance of a populaion appears o saurae, furher increasing he brain size does no seem o improve more. This is due o he simple srucure of his version of he game, here is nohing more o gain. Recall ha only mos crude informaion is ransmied o he players, i.e. only yes and no, no he exac aendance number. More precise informaion would necessiae more analysing power, more complicaed payoff funcions and games also provides incenives o develop more sophisicaed brains. However in he presen work, we sick o he binary funcions and will repor more complicaed applicaions using neural neworks elsewhere Success rae Memory FIG. 2. Success rae of a mixed populaion players agains heir memory (N = 1001, S = 5) The advanage of he larger brain sizes over he smaller ones can be beer appreciaed inspecing Fig. 2. Idenical parameers (N = 1001, S = 5) for a mixed populaion having M = 1,,10. We hus FIG. 3. Hisograph of he aendance of (N = 1001, M = 8, S = 5) Of course he game is symmerical for and B. This can be observed in Fig. 3, where he hisograph shows he aendance 3

4 of (hence B is he mirror image a he poin N = 501). B. rhur s El Farol problem uses 60% rule and does no rise new quesions and resuls appear o be similar. One may argue ha our payoff funcion is oo simple, i.e. a sep funcion wihou differeniaing a good minoriy from a bad one. Le us consider he payoff funcion N/x 2, i.e. hese many (neares ineger values) poins awarded o every player choosing he minoriy side, he number of winning players being x < N/2. Clearly his srucure favours smaller minoriy. This is like in loery you would like o be on he winning side, bu even beer you are alone here. The players hus face an exra ype of compeiion, a winner would prefer less fellow winners in company. If for insance a player wins on a mediocre play, his winning sraegies are hardly enhanced wih respec o no winning a all. Globally he populaion (N = 1001, M = 4) respond o having a hisograph Fig. 4 wih wo peaks. lhough he jackpo (winning alone) is very appealing, his is very unlikely o happen since he fellow players are jus as inelligen. The players need a sizeable gain o ge moivaion o win. The appears o be a compromise ha hey effecively (no hrough any enforceable agreemen) agree o show up on he minoriy side a smaller number of players. Wha is remarkable here is ha enropy, i.e. he mos likely configuraion, does no favour he Fig 4. disribuion. The players manage o defy enropy, in oher words o ge hemselves organised o occupy less unlikely configuraions. One may inquire wha happens if he players are provided wih a bigger idea bag wih more alernaive sraegies. In Fig. 5 we show he resuls for various populaions (N=1001, M = 5) wih S = 2, 3,, 9. We see ha in general wih increasing number of alernaives he players end o perform worse. Wha happens is ha he players end o swich sraegies ofener and more likely o ge confused, i.e. some ouperforming sraegy may disrac he player s aenion, afer being chosen urns ou o be underperforming. We recognise ha his has also o do wih he observaion ime, currenly a player swiches immediaely if anoher sraegy has one virual poin more han ha in use. If a higher hreshold is se, hen he hinderance by increasing number of alernaives can be in par avoided. In he neural nework version of our game, jus one nework (wih adjusable weighs) is given o a player. Le us recall ha in a recen sudy, Borkar e al [5] have proven ha in an evoluionary game players end o specialise in a single sraegy, even hough alernaives exis. Success rae Number of sraegies FIG. 5. Success rae of he bes, wors and average players agains he number of sraegies (N = 1001, M = 5) FIG. 4. Hisograph of he aendance of for a N/x 2 payoff (N = 1001, M = 4, S = 5) 4

5 0.50 poor appears o increase linearly wih ime, hough reversion is possible bu he poor players in general are doomed o say poor. Swiching rae Success rae FIG. 6. Swiching rae agains he succes rae for various populaions Success rae In Fig. 6 we plo he swiching rae agains he succes rae for various populaions. The general endance ha he ofener one swiches, less successful one would end up. The phase space seems o be highly fragmened and many subsrucures appear, aribuable o he binary naure of our game FIG. 7. Performance record of he 3 bes, he 3 worse and 3 randomly chosen players (N = 1001, M = 10, S = 5) I is also insrucive o follow he performance record. In Fig 7., we selec 3 op players, 3 boom players and 3 randomly chosen players. They are chosen a he las ime sep and we race back heir pas record. Their capial gains are scaled such ha he average gain (over he populaion) appears in an almos horizonal line. We see ha he general endance for he bes and wors players are raher consisen even hough sebacks for he bes and burss for he wors do occur. Noice ha he gap beween he rich and he FIG. 8. Differen disribuions of he average value of all he sraegies wih increasing ieraions numbers (1000, 5000 and 10000), showing ha all sraegies are equivalen in he limi noher resul enhances his conclusion : one may blame bad players for heir bad sraegies. In order o check wheher here are really good and bad sraegies, we plo he virual gains of all he sraegies in he populaion. In Fig 8 we see hree differen disribuions of he average (ime) gains. The longer he ime he more concenrae is he disribuion, indicaing ha relaive values of he sraegies are abou he same. Indeed i can be analyicaly shown ha all he sraegies are equivalen o each oher, since our game is symmerical in and B. So he bad player are bad because hey have used he sraegies inopporunely and are unlucky, also heir specific composiion is o blame. Noe ha a player is only disinguished from ohers by his composiion, if wo players having he same composiion, hey are clone sisers. In ha case iniial condiions can sill se hem apar and hey may know differen forunes only in he beginning. The above discussion calls for a geneic approach in which he poor players are regulary weeded ou from he game and new players are inroduced o replace he eliminaed ones. This geneic approach is already applied o a prooype model [6] for sock or currency rading, mimicking wha happens in he real marke. Le us consider 5

6 our minoriy game generalized o include he Darwinis selecion : he wors player is replaced by a new one afer some ime seps, he new player is a clone of he bes player, i.e. i inheris all he sraegies bu wih corresponding virual capials rese o zero. This is analogous o a new born baby, hough having all he predisposiions from he parens, does no inheri heir knowledge FIG. 10. Temporal aendance of of an pure populaion FIG. 9. Temporal aendance of for he geneic approach showing a learning process To keep a cerain diversiy we inroduce a muaion possibiliy in cloning. We allow one of he sraegies of he bes player o be replaced by a new one. Since sraegies are no jus recycled among he players any more, he whole sraegy phase space is available for selecion. We expec his populaion is capable of learning since self-desrucive, obviously bad players are weeded ou wih ime, fighing is among soo-speak he bes players. Indeed in Fig 9 we observe ha he learning has emerged in ime. Flucuaions are reduced and sauraed, his implies he average gain for everybody is improved bu never reaches he ideal limi. Wha would happen if no muaion is allowed and cloning is perfec? Evenually populaion is full of he clone copies of he bes player, each may sill differ in heir decision since he virual capials in heir idea-bag can be differen. In Fig. 10 we plo he performance of such a pure populaion, here appears remendous wase and all srange hings go loose. Indeed, he resuls from inbreeding look raher incesous. s he las experimen we sar he populaion very simple-minded, say M = 2. We allow in he cloning process menionned above an addiional feaure ha a bi of memory can be added or subsraced for he cloned new player, wih a small probabiliy. We wan o be sure ha he rules are such ha his srucural muaion is sricly neural, i.e does no favour bigger brains over he smaller ones, we leave ha o he invisible hand of evoluion o decide. Indeed somehing remarkable akes place: in Fig. 11 we plo he average brain size in he populaion sared wih M = 2, for a populaion of N = 101 and N = The emporal record shows ha here is an arm race among he players. We know by now ha he more brain power leads o advanage, so in he evoluion of survival-of-he-fies he players develop bigger brains o cope wih ever aggressive fellow-players. However such an evoluion appear o saurae and he arm race o sele a a given level. The sauraion values are no universal, having o do wih he ime inervals of reproducion. In general he larger brains need longer ime o learn. Larger populaion (N = 1001) needs more powerful brains o susain he apparen equilibrium han he smaller populaion (N = 1001), also he learning rae (he slope in Fig.11) is smaller. We menion en passan ha populaion s brain sizes do no concenrae on one value, only average value is ploed. Some players manage o make do quie happily wih a relaively small brains. 6

7 Memory average FIG. 11. Temporal record of he memory average of a saring from M = 2 populaion for N = 101 (a) and N = 1001 (b) (S = 5) Conclusions. Wha can we learn from hese simple numerical experimens? Firs of all he economical behavior in he real world seems o call for a general approach o sysemaically sudy he evoluionary naure of games. There are a few mos relevan quesions o address: 1) given each agen s selfishness wha is his cooperaive and cogniive skills in he course of compeiion? 2) Wha is he emerging collecive behavior ha is he sociey s performance wihou an enforceable auhoriy? 3) How can our visible hand modify he rules of he game (payoff funcions) such ha global response may appear more cooperaive? 4) How does evoluion pus is invisible hand o work? Clearly our sudy is jus a beginning o answer all hese. Wha we have presened in his work is no jus an oversimplied model, bu a general approach o ask he righ quesions. This approach, as he reader can readily convince himself, is very open o all sors of variaion. I is easy o include oher siuaion-moivaed payoff funcions and game srucures, here are qualiaively new quesions o be asked when more realisic games are sudied. I is a heoreical physicis s dream o have an Ising ype model, hough oversimplified, and ye o capure some essenial poins of he real world. Our minoriy game may be indeed he simples of he kind. Our model is by design wihou fundamenals and insider informaion. Players are forced o figh each oher. Wih he Darwinism included, everyone has o keep improving in order o survive he red queen effec. Unlike some examples in sandard game heory, here is no commonly acceped opimal sraegy (analogous o physical sysems wihou obvious ground a b saes). raional approach is helpless here. Ye he emerging sociey appears o have a cerain organisaion. Even hough he players care only heir own gain, cooperaion and iming does seem o sponaneously arise. Noe ha our learning mechanism is differen from he radiional neural nework sudies, where a pre-assigned ask like a paern is given and performance is measured on how precise he original is resored. Here he ask is self-appoined and no ending is defined. We may even speak of he emergence of inelligence. If he analysing power of he players can be adaped o he increasingly challenging ask (survival amongs ever aggressive fellow players and larger number of players), he populaions seem o evolve more equipped, larger brains appear o dominae and available resources are beer exploied, i.e. less flucuaion and wase in aendance number. This is no unsimilar o he sudy of he prebioic evoluion: in he promodial soup only very simple organisms exis. Evoluion allows hese organisms o add one new feaure (or reduce an exising one) from ime o ime. More complex organisms end o cope wih he survival ask beer and more and more refined organisms sponaneously appear ou of he monoonous soup [7]. We hank Maeo Marsili for helpful conversaions. This work has been suppored in par by he Swiss Naional Foundaion hrough he Gran No References : [1] The Economy as an Evolving Complex Sysem, ed. P.W. nderson, K. rrow and D. Pines, Redwood Ciy, ddison- Wesley Co.,1988 For a recen example [2] J. W. Weibull, Evoluionary Game Theory, MIT Pres, Cambridge, 1995 [3] M. Marsili and Y.-C. Zhang, Flucuaions around Nash Equilibria in Game Theory, Physica, o appear [4] W. B. rhur, Inducive Reasoning and Bounded Raionaliy, m. Econ. ssoc. Papers and Proc. 84, pp , 1994 [5] V. S. Borkar, S. Jain, G. Rangarajan, Dynamics of Individual Specializaion and Global Diversificaion in Communiies, preprin, Bangalore, IISc-CTS-3/97 [6] G. Caldarelli, M. Marsili and Y.- C. Zhang, prooype Model of sock ex- 7

8 change, Europhysics Leers, o appear [7] Y.-C. Zhang, unpublished; Quasispecies Evoluion of Finie Populaion Phys. Rev. E 55, pp. R3815,