Sensitivity Analysis of Drivers in the Emergence of Altruism in Multi-Agent Societies

Transcription

1 Sensitivity Analysis of Drivers in the Emergence of Altruism in Multi-Agent Societies Daniël Groen Bachelor thesis Credits: 18 EC Bachelor Opleiding Kunstmatige Intelligentie University of Amsterdam Faculty of Science Science Park XH Amsterdam Supervisor Dhr. Dr. B. Bredeweg Informatics Institute Faculty of Science University of Amsterdam Science Park XH Amsterdam June 29th,

2 Abstract This research investigates the effects of age restrictions, reputation, and leeching on the size and social development of a simulated huntergatherer society. The results of this research provide insight into the effects of these features on the social evolution of agents, which contributes to the goal of machine ethics to construct a learning process that produces social behaviour in artificial agents. Previous research has concluded that teaching artificial systems fundaments of social behaviour will increase their flexibility in safely handling complex situations involving moral decision making, and demonstrating these properties can provide an increased level of trust from the general public in autonomous systems. The experiments conducted in this research have indicated that the societies develop a highly altruistic behaviour, and manage to sustain themselves only when the (µ, σ) of the hunting activity reward function are high, combined with a low invitation chance in the first two models. Furthermore, the introduction of age restrictions requires societies to develop optimal resource exploitation strategies to survive. Reputation caused societies to fluctuate highly in size and in certain cases develop shifts in strategies. Finally, the leeching model did not manage to produce sustainable societies. 2

3 Contents 1 Introduction The Field of Machine Ethics Research Overview Approach Simulation Methodology The World Activities and Resources The Agents Variables and DNA Performing Activities Reproduction The Models Basic Group Survival Age Restrictions Reputation Leeching Experimental Setup Results and Analysis Basic Group Survival Age Restrictions Reputation Leeching Conclusion 26 5 Discussion 27 6 Acknowledgements 29 7 Bibliography 30 8 Appendix - Schematic Overview Basic Group Survival 31 3

4 1 Introduction 1.1 The Field of Machine Ethics A recent development within the field of Artificial Intelligence is Machine Ethics. Reasoning about the ethical realm aims to protect humanity and the rest of the world by applying constraints to the capabilities of robots and computers, in order to reduce unforseen consequences through reaching uncontrollable states as well as through misuse [1]. Machine Ethics has evolved from the interactions between the disciplines of Artificial Intelligence and Ethics, and is concerned with creating computer controlled artificial agents that are cooperative and altruistic and can be safely integrated into human societies [2]. Technological advances have given birth to sophisticated machines with a wide range of capabilities. With this growing power comes an increased requirement for autonomous control in order to apply these capabilities safely [1]. Moreover, once artificial agents manage to present their capacity to sucessfully handle social challenges, the general public could potentially become more comfortable with the integration of these autonomous agents [3]. Machine Ethics aims to investigate the key features that can be incorporated into agents that will cause them to emit social behaviour. Most research thus far has been conducted from a top-down perspective on ethics, which relies heavily on the assumption of an established theory of ethics which is directly translated into algorithms and behavioural rules. However, a bottom-up approach may prove to be more viable. By incorporating the key features into a machine learning process, the goal is that the agents inevitably discover that cooperation and social behaviour is the most rewarding behaviour [4]. Researchers have investigated human moral development in order to acquire insight about which features stimulate this. Humans have developed a high capacity for social behaviour, allowing them to engage in complex social interactions, which researchers believe is fundamental to developing moral behaviour in general [3, 5]. 1.2 Research Overview The goal of this research is to acquire insight into the effects of certain features, conditions and mechanics on the social development of an artificial agent society. This is done by conducting a series of simulations which simulate the evolution of hunter-gatherer societies. These societies form the basis of modern human development and span the majority of human evolution. Studies have indicated that complex structures of our social capacities are present in these societies, therefore, modelling their evolution may provide insight into the features that stimulated this social development [6, 7]. By sequentially adding and changing components of the agents simulation environment, the changes in the results provide insight into the effects and importance of these features in the development of human social capacity, which ultimately provide an understanding of which features to incorporate in a bottomup machine learning approach for constructing social agents. The main question of this research therefore is: What are the effects of Age Restrictions, Reputation and Leeching in a multitude of configurations on the development of altruism in 4

5 a multi-agent society? 1.3 Approach The methodology of this research is in the form of an incremental sensitivity analysis. Research has indicated that great variance existed multiple different hunter-gatherer societies over time, suggesting that a multitude of settings should be tested in order to acquire a more complete image of the social development of hunter-gatherers [8]. First, a basic simulation model is constructed, the Basic Group Survival model, which contains the society and its environment, simulating the most simplistic survival conditions. Generally, the agents venture out daily on two different foraging activities, hunting or gathering, which they perform alone or in groups. Both activities have a different food reward function, and are affected by the size of the group of cooperating agents (see sections 2.2 and 2.3). Hereafter, three consecutive models are built: Age Restrictions Reputation Leeching Each of these models introduces a new mechanic that adds or changes certain conditions from the previous model, incrementally increasing the complexity of the survival process. Furthermore, a sensitivity analysis is performed for a multitude of parameters for each model, which constitutes running the model with a number of altered values, such as the number of agents on initialisation and the food rewards of activities. When running the simulation and sensitivity analysis of a model, the society s data is collected over time and analysed. These data contain the information about the average social ratio of the agents, activity preference ratios, health levels and more. The collected data is described in section Simulation Methodology Four models have been built in Python 1 (version 3.6.2, 32-bit). Each model is run with a number of different parameter values. The following sections first outline the environment that forms the basis for every model. Hereafter, the Basic Group Survival section describes the basis of the simulation process, whereas the subsequent sections focus on the incorporation of their respectively introduced mechanic. Finally, section 2.5 covers the parameters that are altered in the sensitivity analysis as well as the set of values that the experiments focus on. 2.1 The World The world houses the society of agents and the environment that the agents can interact with. Every iteration loop within the world constitutes one day, in 1 5

6 which every agent determines their course of action, perform a single activity, and reproduce if certain conditions are met. The world does not have a spatial dimension, i.e. the agents are not located in a grid-like environment. Instead, the interactions are based on conditions and chances. 2.2 Activities and Resources The activities are the source of resources, or food, for the agents, which allows them to maintain their health. Each day, every agent loses 25 health points, modelling fixed daily energy requirements in for example homeostasis. In total, this means that the agents can survive for 40 days before starving and being removed from the world, assuming that there is an abundant supply of water available [9]. The agents attempt to combat this continuous energy requirement by engaging in foraging activities. There are two activities that the agents can perform - hunting and gathering. Hunting represents the act of acquiring meat from animals, whereas gathering represents searching for plants and other edible vegetation, both having separate availabilities within the environment. The rewards of performing these activities are based on chance - the available resources for each activity are not defined spatially (in accordance with the absence of a grid-like environment), instead they have a predetermined availabilities which constitute the maximum amount of resources that can be acquired daily by the society. These availabilities are replenished daily by certain amounts (see section 2.5), and the environment has a predefined maximum resource capacity of 5 times the daily replenishment value for both activities. Every day, each agent can perform a single activity - either hunting or gathering. In the process of choosing the activity, the agent decides whether to cooperate with other agents, or forage alone. Cooperating is done by inviting other agents into a group that forages together, and this group will receive a reward that depends on the activity chosen by the inviting agent combined with the size of the particular group. The invitation process is described in section How much resources are acquired through performing an activity is determined by the reward function of the activity. Both activities have their own reward function that returns the amount of resources every agent in a group of cooperating agents receives, according to their group size. These groups can consist of a single or of multiple agents, meaning that agents foraging alone are treated as a group with a size of 1. Every agent in a group receives the same reward. The gathering activity has a scaling function, which provides a per agent reward that diminishes with increasing group sizes. Formula: r = b size 1.6 The theory behind this formula is that a group of agents focusing their search on one area should on average find less per agent, compared to each agent searching a different area. In the latter case, in this simulation, the agents are considered to be in separate groups which are not cooperating with each other. The specific value has been determined iteratively, by running simulations with a number of different values. 1.6 is sufficient for a sustainable society (see the Discussion section). The base of the reward, b, will be adjusted throughout the experiments according to the survivability of the society. For example, the second model introduces childcare (see section 2.4.2), which will increase the resource strain on the agents. In this case, they will need resources to not only 6

7 support themselves but other agents as well. Figure 2.1 illustrates the per agent resource returns against the daily energy requirement of an agent: Figure 2.1. Visualisation of the Gathering activity reward function against the daily energy requirement The hunting activity uses a normalised gaussian distribution function, which focuses on an optimum group size. Groups which are of this size will receive the maximum reward, diminishing gradually as the group size deviates further from this optimum value. size)2 (µ Formula: r = b e 2σ 2 1, where µ and σ determine the 0.2 size1.1 mean and spread of the reward curve, respectively. Multiple combinations of (µ, σ) are tested per model. The first part of the formula contains a normalised version of the gaussian formula so that the curve always scales up to the maximum reward b. The second part of the formula is a cost function, modelling an effort requirement depending on the group size. The theory behind this formula consists of two parts. Firstly, hunting alone significantly decreases the success chance when hunting larger prey or groups of prey, forcing an agent to focus on smaller prey with lower resource values. With larger groups, catching larger prey and groups of prey becomes more feasible, allowing higher per agent rewards [10, 11]. However, hunting in too large groups means dividing the rewards among a high number of agents, resulting in decreased per agent rewards. The different combinations of (µ, σ) are tested in order to determine the effects of different hunting group size requirements on the cooperation within the society, representing the different hunting availabilities throughout different environments. Likewise, this formula is constructed using an iterative approach to approximate the underlying theory while providing a sustainable society. Figure 2.2 shows the curves for the (µ, σ) value combinations that will be utilised in the sensitivity analysis: 7

8 Figure 2.2. Visualisation of the Hunting activity reward function for multiple (µ, σ) settings against the daily energy requirement 2.3 The Agents An agent constitutes an individual within the society, which makes decisions about which activities and social actions to perform. The agent s decisions are determined by its DNA, which will evolve throughout the simulation as the society survives and reproduces. A genetic algorithm is used in the evolution process, creating new agents with different DNA compositions that allow the society to evolve towards an optimal survival strategy. These values are additionally acquired to assess the state of the society during the simulation, which are used in the analysis of the models Variables and DNA Every agent that is initialised into the world, at the start of the simulation as well as through reproduction, receives a number of values that determine its characteristics, defining for example its health and DNA. The DNA of the agent consists of the altruism level and activity ratio, which are used in its decision making process as well as in the collection of statistics throughout the simulation. The following variables are included: Health (0-1000) - Constitutes the fitness of the agent, health is accumulated by acquiring resources, and spent as energy investments in performing activities and reproduction, and is decreased by 25 every day (see section 2.2). Every new agent is initialised with maximum health(1000), and will be removed from the world once its health reaches 0. Age (0-70) - Keeps track of the agent s age in simulation years according to the amount of days (iterations) it has survived. One year is equal to 36 iterations 2. 2 Aging is increased by approximately tenfold in order to increase the rate of convergence in the simulations. 8

9 Altruism ( ) - Constitutes the social level of the agent, it determines how likely the agent is to perform a cooperative action. In performing activities, the agent can either choose to be cooperative and try to forage with other agents, or to be egoistic and forage alone, the ratio of which is determined by this value. A higher altruism level increases the chance that the agent performs an activity cooperatively (see section 2.3.2). This is the first part of the agent s DNA - the agent uses this chance in deciding whether to cooperate with other agents or to act alone in performing an activity. This value is used as the measure of sociality within the society. Activity ratio ( ) - Determines how likely the agent is to choose the hunting activity over the gathering activity when initialising an activity. This is the second part of the agent s DNA. The agent has separate activity ratios for both the egoistic and the social actions, meaning it can develop a different activity preference for cooperative actions than for egoistic actions Performing Activities Every agent uses its DNA in the process of selecting and performing activities. There are 2 choices that an agent can make based on its DNA. The first choice is selecting an action - cooperative or egoistic. A uniform random probability is generated and compared to the Altruism level of its DNA - if this probability is above this altruism threshold, the agent selects the egoistic action, and likewise if the probability is equal to or below the altruism threshold, the cooperative action is selected. The second choice is selecting an activity, which is similar in functionality. Another random probability is generated, which is compared to the value of the Activity gene corresponding with the selected action. If this probability is above this activity threshold, the Gathering activity is selected. Otherwise, the Hunting activity is selected. In every iteration, each agent forages once. The simulation randomly selects an agent from the society, and this agent takes a number of steps. First, the agent selects an action and an activity. If the egoistic action is selected, the agent then performs the activity as a foraging group of size 1. This behaviour is altered in the Leeching model (see section 2.4.4). Afterwards, the agent can no longer participate in any foraging activities for the duration of the iteration. However, if the cooperative action is selected, the agent attempts to invite other agents into its foraging group. Every other agent in the society is sent an invitation depending on a predefined invitation chance (this mechanic is adjusted in the Reputation model, see section 2.4.3). Every agent that receives this invitation will respond by either joining the foraging group or rejecting the invitation. If the receiver can no longer perform an activity in the current iteration, the invitation is rejected. The receiver decides by selecting its own action. If this is the cooperative action it joins the foraging group, otherwise it rejects the invitation and does not join the group. The chosen activity is not taken into account in this decision, the receiver will join or reject regardless of the activity decided on by the inviting agent. Once the foraging group has been established, every group member receives the 9

10 reward computed by the reward function of the respective activity. Furthermore, every agent that has participated in the foraging group, including the inviting agent, cannot participate in other foraging activities during the iteration Reproduction At the end of every iteration, each agent checks whether it meets specified reproduction criteria, and if such is the case, it searches for another agent that meets these criteria as well. This is done by comparing certain variables with the reproduction conditions: Age - The agent s age is between 20 and 35, in between which it has the ability to reproduce [12] Health - The agent has at least 500 health points remaining to accomodate for the energy requirement Last Reproduction - The agent has not reproduced in the last 4 years [13] Following a succesful reproduction, a new agent will then immediatly be initialised within the society. First, this agent receives the average DNA of both of its parents: Altruism Activity ratios - Cooperative Action and Egoistic Action Subsequently, this DNA receives a mutation - each of the variables will either increase or decrease by a default mutation rate of Each variable mutates separately and increases or decreases with equal probability. The mutation rate is altered in certain simulations if the society is unable to adapt at a sufficient speed in order to survive. The age of the new agent is set to 0 and its health is set to Finally, both parent agents have 100 points deducted from their health as an energy requirement of the reproduction. 2.4 The Models Basic Group Survival This section describes the basis underlying every consecutive model. At the start of the simulation, the world is created and initialised with a number of agents, and the resource availabilities are set to their respective daily replenishment values. Afterwards, the simulation will loop for a specified number of iterations, or until there are no more agents left. In each iteration, every agent is deducted the daily energy requirement of 25 health points and has its activity participation limit reset to 1. If an agent s health has reached 0, or it has reached the maximum age of 70, the agent is removed from the list of agents. Afterwards, the world iterates sequentially through the list of agents. Each of these perform an activity as described in section and add the acquired resources to their health level. Then, each agent enters the reproduction procedure, validating whether it meets the criteria and finding another agent if it does. The daily resource replenishment values are added to the resource availabilities of their respective activities, up to the maximum capacity of the environment. 10

11 2.4.2 Age Restrictions This model builds directly onto the Basic Group Survival model, and incorporates the concept of children and childcaring mechanics. A commonly employed strategy for childcare in hunter-gatherer societies was non-parental caregiving. Contrary to the dominant strategy in current western societies, the community as a whole would engage in the task of raising the young in hunter-gatherer societies [14]. This model introduces an age category containing agents that are below the age of 12. These agents cannot yet participate in activities, and therefore cannot acquire resources to maintain their own health level. Instead, the society maintains a resource storage from which the children can acquire their required resources. This storage will be supplied by the rest of the society after performing activities, depending on their altruism levels. Once an agent receives a resource reward, it selects an action based on its altruism level in its DNA. If this is the cooperative action, the agent contributes 30 percent of its rewarded resources towards the storage, replenishing its own health with the remaining 70 percent. Otherwise, the agent does not contribute resources towards the storage, instead it replenishes its own health with the full resource reward. At the end of every iteration, every agent below the age of 12 takes resources equal to its daily energy requirement of 25 from the storage, as long as the storage has sufficient supplies. If any of these agents has a health value of 600 or below, it takes 10 additional resources to try and maintain a decent health value Reputation This model incorporates a reputation mechanic into the Age Restrictions model, which alters the cooperation invitation process. Here, the agent s altruism levels are used instead of a static predefined invitation chance. The altruism level constitutes an agent s reputation within the society, and every other agent can access this value. An agent that has selected a cooperative action will now invite agents based on their altruism level, meaning that having a higher altruism level increases the chance of being invited into foraging groups Leeching In this final model, leeching is introduced into the Reputation model. This resembles joining foraging groups and claiming a share of the resources, without providing cooperative effort. This changes the cooperation process and the rewards that groups receive from activities. In receiving a cooperation invitation, an agent has to respond by selecting either the cooperative or the egoistic action. This model alters the process of when an agent selects an egoistic action in the response. Instead of rejecting the invitation, the receiver will now join the foraging group as a leecher, while the agents that chose the cooperative action are considered as cooperators. The foraging groups will receive a reward based only on the amount of cooperators that are present in the foraging group, however, the reward is divided among all of the agents present in the foraging group. This means that the reward function returns a total reward based on the amount of cooperators, which is multiplied by the amount of cooperators but divided by the total number of agents in the group. Therefore, the leechers 11

12 will receive a reward while not contributing to the total resource reward, consequently decreasing the per agent reward of a group when leechers are present. 2.5 Experimental Setup A number of simulation parameters have been selected for the sensitivity analysis: 1. Initial Agent Count - (30, 50, 80, 100) The initial agent count is potentially influential in the capacity of the society to adapt towards an optimal resource exploitation strategy such as constructing optimally sized foraging groups, by having an increased reproduction capacity at the beginning of the simulation 2. Hunting Activity (µ, σ) - ((6, 4), (10, 6), (15, 8), (25, 12)) These determine the viable sizes of the hunting activity groups. Since the formation of these groups is dependent on the society s altruism level, these parameters are likely to affect the development of the society (see Figure 2.2 for a visualisation of these parameter values) 3. Invitation Chance - (100, 80, 60, 40) Used in the models Basic Group Survival and Age Restrictions. This is a significant driver of the foraging group sizes in these models. Changing this value potentially impacts the altruism level of the society in order for it to construct optimally sized foraging groups. 4. Resource Replenishment - This is set to (1000, 1500) for gathering and hunting respectively by default. This is adjusted to (1500, 1000) in situations where the society manages to exploit one or both of the activities resource availabilities, in order to measure the effect of different environmental resource availability ratios. Hunter-gatherer societies lived in society sizes of approximately a few dozen, as such the total daily resource replenishment is chosen at 2500, which is sufficient to sustain societies of approximately 60 to 80 agents [10]. The mutation rate used in the reproduction process is set to 0.05 by default, meaning for each value in an agent s DNA, a mutation will either add 0.05 to that value or subtract 0.05 from it. This mutation rate is increased when a specific combination of parameter values results in the society consistently going extinct, in order to investigate whether the society can survive with increased adaption capabilities. The following data is collected during the simulation to determine convergence in cooperation policies and to analyse the effects of new mechanics and parameter changes: The average health level of the agents The average altruism level of the agents The average activity ratios of the agents The average count and size of foraging groups The society composition of different age categories 12

13 The amount of remaining resources in the environment, indicating the effectiveness of the resource exploitation strategies of the society A large number of parameter combinations are tested. The strategy that is used here involves testing the full range of a single parameter in order to determine its effects and viable ranges. If a parameter settings for example appears not to influence the simulation or to consistently drive the society to extinction, that specific value is excluded from further testing. This greatly reduces the amount of combinations, providing a more compact overview of their respective effects on the simulation. 3 Results and Analysis The graphs in the following sections illustrate the data collected about the society, activities and resources throughout a simulation process. The values on the horizontal axis of each graph display the simulation iteration. The vast majority of the simulations shows to converge within 40, 000 iterations, which is consequently set as the standard for a succesful run that posits that the society is able to sustain itself. Simulation runs that terminate before this are considered to have parameter combinations that lead to unsustainable societies. Additionally, if one of the graphs shows that a certain type of data has not converged towards a clear terminal value or is otherwise not fluctuating constantly around an average, the simulation duration is extended and analysed again. The data about a specific combination of parameters is averaged over 5 runs to represent a more accurate image of the optimal development. Finally, the results of multiple simulations are analysed and compared against each other. Differences between the simulation results are highlighted and outlined using several visualisations. Each of the 6 types of data is visualised using graphs, averaged per 100 iterations: Average Health Level - Displays the average health of the agents in the society. Altruism Level - Displays the social ratio of the agents: blue represents the altruism level defining the cooperative action chance, and orange shows the remaining chance of the egoistic action. Activity Ratios - Shows the chance of the hunting activity in blue against the gathering activity chance in orange, for both the cooperative action and the egoistic action. Groups - Displays the average amount of groups in the first graph and the average size of these groups in the second. Society Composition - Shows the size of the society and the composition of the age categories. In the Basic Group Survival model, three age categories exist: (0-19, pre-reproduction age) in blue, (20-35, within reproduction age) in orange and (36-70, after reproduction age) in green. In the consecutive models, the category (0-19) is divided into two separate categories: (0-11, children) in blue and (12-19, able to perform 13

14 activities but cannot yet reproduce) in orange, along with the categories (20-35, within reproduction age) in green and (36-70, after reproduction age) in red. Resource Availabilities - Displays the average amount of resources remaining in the environment at the end of an iteration along with the maximum amount of resources supported by the environment indicated by the horizontal blue line. The remaining resources for the hunting activity are shown in the first graph, and for the gathering activity in the second. 3.1 Basic Group Survival First, the effects of the Initial Agent Count parameter is tested. Four simulations are run with each of the Initial Agent Count values (30, 50, 80, 100), using only the first values of the Hunting Activity, Invitation Chance and Resource Replenishments value configurations, being (µ = 6, σ = 4), (100) and (1000, 1500) respectively. Furthermore, the value of b in the reward functions is set to 35 for the hunting activity and to 29 for the gathering activity. Each of these simulations resulted in an extinction within approximately 3000 iterations (2-3 generations), even with an increased mutation rate. This consequently means that there is insufficient data to determine whether the society is developing towards a specific strategy. The simulations are then rerun with the Invitation Chance adjusted to its second value configuration (80). Similarly, none of these four simulations were successful, however the average amount of iterations increased to approximately At an Invitation Chance of (60), the simulations ran for between 14, 000 and 27, 000 iterations. Figures 3.1 through 3.6 show the collected data of the simulation run with the following parameter values combination: 1. (50), 2. (µ = 6, σ = 4) and 3. (60). 14

15 Figure 3.1. Social Ratio Figure 3.2. Activity Ratios Figure 3.3. Groups Figure 3.4. Society Composition Figure 3.5. Resource Availabilities Figure 3.6. Average Health Level By investigating the remaining resource amounts in figure 3.5, it becomes clear that the society has not succeeded in evolving a strategy to exploit the resources effectively, preventing the society to grow. Running the four simulations with the Invitation Chance adjusted to 40, the society succesfully manages to survive. 15

16 Figure 3.7. Social Ratio Figure 3.8. Activity Ratios Figure 3.9. Groups Figure Society Composition Figure Resource Availabilities Figure Average Health Level The decreased Invitation Chance parameter here causes the agents to form smaller groups. This is beneficial for gathering as well as hunting, since the optimum hunting group size is significantly smaller than the average group sizes in earlier simulations when the society has more agents, This is shown in figures 3.13 and 3.14, from the simulation with parameters combination (30), (µ = 6, σ = 4) and (80) respectively. 16

17 Figure Society Composition Figure Groups However, the society still shows to be unable to exploit the environmental resource availabilities effectively. Furthermore, the development of the society appears to have become very decisively altruistic. A possible explanation for this is that cooperating with more than the optimum amount of agents in the hunting activity provides higher rewards compared to performing this activity alone. The average data throughout the simulation appears not to be affected by changing the Initial Agent Count, as long as the society is able to form sufficiently large foraging groups at the start of the simulation. Therefore, this parameter is discarded in further sensitivity testing and set to a static value of (60). With this value, the society is still able to grow with respect to the resource availabilities, yet it still has a sufficient initial size to construct larger foraging groups. The next simulations investigate the effects of lowering the Invitation Chance along with increasing the Hunting Activity parameters. This is done by simulating each remaining Hunter Activity parameter value, ((µ = 10, σ = 6), (µ = 15, σ = 8) and (µ = 25, σ = 12)), with every Invitation Chance value. At (µ = 10, σ = 6) in combination with (100), the society is not able to survive for 40, 000 iterations. Additionally, the society again develops to be highly altruistic, similar to figure 3.7. However, in this simulation the society evolves a decisively higher hunting activity ratio for the cooperative action. This is an effective strategy, since the average foraging group size approximates the optimum hunting group size parameter, providing near-optimal rewards. This is illustrated in figures 3.15 and

18 Figure Activity Ratios Figure Groups The following simulations show results similar to the previous test - a high Invitation Chance causes the society to decrease in size considerably. However, with a Hunting Activity value of (µ = 15, σ = 8) and (µ = 25, σ = 15) the society is now able to sustain itself and prevent exctinction. The Hunting Activity parameter is therefore capable of preventing excinction in simulations with a high Invitation Chance, since the scaling is more suited to the larger foraging groups that result from the high Invitation Chance value. One additional configuration is tested, using a new Hunter Activity parameter value of (µ = 35, σ = 15) and an Invitation Chance of (40) to further investigate the resource exploitation capacity of this model. The society in this simulation, in contrast to the previous simulation, shows to exploit the hunting resource availabilities significantly better than the gathering resources. Although the activity ratios average around 0.5 for both actions, the exploitation shows to have a positive impact on the size of the society. These data are illustrated in figures 3.17 and Figure Resource Availabilities Figure Society Composition Concluding, the Invitation Chance along with the Hunting Activity parameters are the most influential on the sustainability of the society. The former affects the size of the foraging groups, whereas the latter determines the most 18

19 effective group size for the hunting activity. The optimum intersection point appears to be at an Invitation Chance of (40), with a Hunting Activity parameter value of (µ = 35, σ = 15), where the society is able to exploit the environment optimally and reach its maximum size. 3.2 Age Restrictions In this model, none of the configurations used in the previous section provided a sustainable society. However, this is an expected result, since the resource strain on the society is increased as a consequence of taking care of their young. By increasing the resource base rewards b of hunting and gathering to 44 and 38 respectively, certain configurations are able to produce sustainable societies. The first configuration that survived for 40, 000 iterations contains the Hunting Activity parameter value (µ = 15, σ = 8) and an Invitation Chance value of (60). However, the resulting data shows that this model requires more iterations to converge, specifically the altruism level of the society, which is shown in figure Figure Social Ratio The model is therefore simulated for a total of 60, 000 iterations, which subsequently does appear to converge. In contrast to the Basic Group Survival model, this simulation shows that the agents have reached the capacity to exploit the available resources completely. This in turn has allowed the society size to increase. Figures 3.20 through 3.25 illustrate the collected data. 19

20 Figure Altruism Level Figure Activity Ratios Figure Groups Figure Society Composition Figure Resource Availabilities Figure Average Health Level The model is additionally run with an alteration of the Resource Replenishment ratio, which is changed from (1000, 1500) to (1500, 1000), meaning that the environment instead contains a larger amount of resources in the gathering activity than the hunting activity. Figures 3.26 through 3.31 display the results of this simulation. 20

21 Figure Altruism Level Figure Activity Ratios Figure Groups Figure Society Composition Figure Resource Availabilities Figure Average Health Level In conclusion, the Age Restriction model shows to increase the resource exploitation capacity of the society. The agents require a larger amount of resources in order to maintain their young, possibly forcing them to evolve the most effective society-wide strategy in order to prevent extinction. In addition, the agents appear not to develop the decisively altruistic behaviour as consistently as in the previous model. 21

22 3.3 Reputation The Reputation model is first tested with the lowest Hunting Activity parameter value that results in a sustaining society, being (µ = 10, σ = 6), and simulating 60, 000 iterations. This is lower than the Age Restrictions model, which required minimally (µ = 15, σ = 8) in order to achieve a sustainable society The Invitation Chance is not used in this model, since the mechanic of this model replaces this by an agent s reputation, or altruism level. With this setting, the society also managed to survive. The data is displayed in figures 3.32 through

23 Figure Altruism Level Figure Activity Ratios Figure Groups Figure Society Composition Figure Resource Availabilities Figure Average Health Level In contrast to the Age Restrictions model, the society in this model converges quickly to very high altruism levels, similar to the Basic Group Survival model. Additionally, this parameter configuration has reached the highest cooperative hunting ratio compared to previous simulations, as well as containing the highest fluctuations in society size and average health level. Since the invitation chance in this model is instead dependent on the altruism 23

24 degree, a shift in behavioural strategy can be seen when the altruism level of the society develops, which is absent in the previous models. Once the level of altruism in the society increases, the probability that agents will forage together in larger groups increases simultaneously. When performing activities in large groups, it is more rewarding to select the hunting activity due to the relatively high Hunting Activity parameter value, which may explain the behavioural shift. This is further supported by the shift in resource availabilities - the society first majorily exploited the gathering resources, but gradually switches to exploiting the hunting resources. Afterwards, another simulation is run with a higher value for the Hunter Activity parameter, at (µ = 15, σ = 8). This model converged within 40, 000 iterations, and the data is shown in figures 3.38 through

25 Figure Altruism Level Figure Activity Ratios Figure Groups Figure Society Composition Figure Resource Availabilities Figure Average Health Level This simulation develops a high altruism level similarly to the previous simulation. However, while the fluctuation in health and society size remains, there is no shift in behaviour. Instead, the society focuses majorily on the gathering resources, and moreover decreases the overal hunting activity rate for altruistic actions. Finally, another simulation is run with a further increased Hunting Activity pa- 25

26 Figure Resource Availabilities Figure Activity Ratios rameter value over the previous simulation. The results resemble the previous simulation, except that there is no decrease in the hunting activity ratio, and the society manages to exploit all of the resource availabilities and maintain a large society size. These are displayed in figures 3.43 and In conclusion, this model appears to develop very high altruism levels, which may be inefficient with low Hunting Activity parameter values. Nevertheless, the society can maintain itself and partially exploit the available resources with these settings, although the society remains relatively small in size. With higher parameter values, the activity ratios remain more constant at approximately 0.5. While their ability to exploit the environment increases with the parameter value, the fluctuation in health levels and society sizes persist. 3.4 Leeching The Leeching model is first tested with the minimal configuration of the Age Restrictions model. With these settings, the society goes extinct within 300 iterations. Since the resource strain on cooperative agents is increased due to leechers, the resource rewards and the Hunting Activity parameter value are increased. However, after tripling the base resource rewards of both activities, the society is still unsustainable. There is a high fluctuation in the number of survived iterations, suggesting that this mechanic destabilises the underlying basis mechanics severely. A possible explanation for this problem may be related to the simplified leeching mechanics. Sustaining leeching requires the implementation of a more advanced leeching mechanic. 4 Conclusion Four models have been built, in order to examine the effects of drivers and parameter settings on the social development of a multi-agent society. These models describe an evolution process of a society that has to sustain itself by acquiring resources and performing reproduction. The first model constitutes the basis of the simulation process, and the consecutive models each add a new layer of complexity into the survival process. Each model is run alongside a 26

27 sensitivity analysis to investigate the effects of the model in combination with different simulation parameter configurations, which is done by collecting data that represent the social development of the society throughout the simulation. The results from the Basic Group Survival model have shown that the sustainability of a society depends greatly on the optimum foraging group size of the hunting activity, where combinations of higher (µ, σ) also lead to larger societies. The average foraging group sizes in the societies are determined by the invitation chance in the Basic Group Survival and Age Restrictions models, where a lower value leads to group sizes that are closer tot the optimum sizes. A combination of a low invitation chance with a high (µ, σ) value allowed the societies to better exploit the resource availabilities, developing increased society sizes. These societies also evolved high altruism levels, which is likely due to the alternative being foraging alone. Since the invitation chance regulates the foraging group sizes, it is plausible that evolving a cooperative nature maximises the amount of foraging groups, providing increased rewards than foraging alone. Furthermore, high initial agent counts appeared not to influence the development of the societies. The Age Restrictions model required higher base resource rewards for the activities, and showed to exploit the resource availabilities more than the previous model with a low invitation chance and high (µ, σ) values. Here, these societies went extinct in configurations where they developed a strategy that could not exploit the availabilities. This suggests that the increased resource strain from childcare requires a society to cooperate in order to acquire the highest possible rewards from the hunting activity, while additionally maintaining a balance between performing hunting and gathering activities. The Reputation model required similar parameter settings to the previous model. However, it showed to converge at a slower rate as well as fluctuate highly in society size. Moreover, some simulations showed a shift in strategy development as the societies increased in altruism level. These societies did not manage to exploit the resources to the extent of the previous model, however they often exploited the resources of the gathering activity completely while intermediatly exploiting those of the hunting activity. Finally, the leeching model appeared not to be able to create conditions for sustainable societies with any configuration as well as tripled base resource rewards. The different simulations did fluctuate highly in the amount of survived iterations, however neither of these managed to sustain themselves. This is most probably due to simplified mechanics of the leeching models. 5 Discussion This research has aimed to provide a broad overview of the effects of multiple drivers on the development of altruism. In this process, the mechanics of each model have been simplified. As demonstrated by the Leeching model, additional interactions and mechanics are required to acquire more accurate information about the effects of these drivers. Future research could focus on one of the mechanics and implement additional layers of interactions. For example, the agents do not consider direct reciprocity in the Reputation and Leeching models, instead reputation is static and determined at birth in DNA, which could be represented by a value that varies according to the actions selected by an agent. 27

28 Modelling a more accurate reputation system may provide insight into the effects of developments of involved brain regions such as memory on social behaviour. In the survival mechanics deployed in this research, the agents are not able to apply cooperation in different ways, and future work for example could expand on this by adding inter-group coordination, where groups split up to cover different areas or activities, rather than focusing on the same ones. Furthermore, this research focuses only on the collection of food. A broader range of activities could be incorporated that for example requires the agents to cooperate by coordinating the distribution of effort among multiple tasks. Additionally, this research separates cooperative actions from egoistic actions completely. An action model could be implemented where outcomes are determined by the degree of altruism of an agent. These expansions would provide more accurate insights of the advantages and disadvantages of altruistic and egoistic action choices. Another interesting possibility for future research is the expansion of the environment and environmental resource mechanics. These are dynamic systems in reality that heavily influence an agent s behaviour and capabilities. The reward functions in this research are approximations of theories about restricted interactions which have been adjusted to provide sustainable societies, and are consequently inflexible and independent of the environment. Increasing the interaction of the resource availabilities with the environment is necessary to provide more accurate information. Additionally, researching the development of altruism in societies located in different environments, such as different weather conditions and available types of resources, may provide insight into diversities between societies. 28

29 6 Acknowledgements I sincerely wish to thank my supervisor Bert Bredeweg for the continued support in constructing the focus of this research. I highly appreciate the structural advice and feedback on the simulation models and on this research paper, which has allowed me to conduct this research to the extent with which I envisioned this project. Additionally, I wish to thank Sander van Splunter for the additional support in structuring and motivating my ideas. 29