BIEB 143 Spring 2018 Weeks 8-10 Game Theory Lab

Size: px

Start display at page:

Download "BIEB 143 Spring 2018 Weeks 8-10 Game Theory Lab"

Silas Neal
5 years ago
Views:

1 BIEB 143 Spring 2018 Weeks 8-10 Game Theory Lab Please read and follow this handout. Read a section or paragraph completely before proceeding to writing code. It is important that you understand exactly what to do in order to successfully write the code. R Concepts Lists A new data type could be useful for this assignment. We have used vectors to store items of the same type (characters or logicals or numbers) and have used arrays as multi-dimensional extensions of vectors. In this assignment, we will be working with collections of vectors of different lengths. These can be tricky to store in an array: how would you make an array out of two vectors, one of length 5 and one of length 10? As an alternative you can use a list. A list is like a vector except that the value of the entry at each position can be anything. So you could have a list that is: L1=list('a',4,TRUE,3,'hi','I am list entry #6',c(3,4,5,1,5)) For our purposes, it is more useful that an entry of a list can be a vector. So you could have this list: L2=list(runif(40),c(4,6,1,6),c(4:50)) This is a list of length 3 and each entry is a vector. To access positions in a list, you use double brackets. So if you typed: L2[[2]] you would get the vector c(4,6,1,6). If you typed: L2[[2]][3] you would get the value 1 because it is the value in the 3 rd position of the vector in the 2 nd position in the list L2. To make an empty list, you can type: alist = list() To assign the vector 1:40 to the Nth position of alist you can type: alist[[n]]= 1:40

2 The next two R commands - NA and rep() - could be useful later on in this assignment if you want to vectorize your code. They won't really be useful before that. NA While a list can contain different types of values, a vector can only contain one type of value. What if you want a vector of length 10 but you only have 7 values to put in it? What do you do with the remaining 3 positions? You can use NA (which stands for 'Not Available') for the missing values. You can use NA no matter if your vector is a character vector, a logical vector, or a numerical vector. All of the following are valid vectors: c(2,4,1,5,6,7,4,na,na,na) c('t',na,'ts',na,'tsp','w',na) c(na,na,na,true) To test to see if an value is NA, you can use the function is.na(): v=c(na,na,na,true) is.na(v[1]) would be TRUE is.na(v[4]) would be FALSE rep Sometimes it is useful to make a new vector that consists of a smaller vector repeated multiple times. rep() does this, and it has some flexibility in exactly how it does it. Let's say you have the vector: x=c(1,2,3) Below are some examples of how to use rep() and the output it would give you > rep(x, times=4) [1] > rep(x, length.out = 8) [1] notice how it dropped the last 3 because it reached length 8 > rep(x, each = 3) [1]

3 times and length.out are alternatives and each can be combined with either. If you want to combine them you should try it out with a small example to see how it works. The Game We will play a game in which a pair of contestants compete for a pool of resources worth k calories. A single encounter in this game is called a bout and is between two players. To play a bout, the two contestants have to decide how many calories they wish to invest into getting the resource. Whichever player invests more wins. For example, if player 1 invests 98 calories (P1=98) and player 2 invests more calories (P2=99), then player 2 wins. Both players pay the minimum of the two plays, and the winner gets the pool of k calories. In this example, let s say k=180. Both players pay 98 calories, and player 2 wins the pool of 180 calories. Here s the calculation for their final payoffs: Payoff to player 1 = = 82 calories Payoff to player 2 = -98 calories Note that players can invest any amount in a bout and payoffs can be negative for both players. For example, if P1=185 and P2=181, then: Payoff to player 1 = = -1 calorie Payoff to player 2 = -181 calories If player 1 and player 2 make the same play, then a tie ensues, and the pool k is split. For example, if P1=200, P2=200, and k=180: Assignment Payoff to player 1 = /2 = -110 calories Payoff to player 2 = /2 = -110 calories Your assignment for this lab is to write a program that will evolve a player that is able to play and win this game. You will then compete your evolved player in a classroom Olympics with fellow lab students. To facilitate and guide you through the evolution, you will be asked to complete the assignment in three defined steps. Taking a stepwise approach will be important because we will face increasingly complex game situations where the optimal solutions are not necessarily known, and where they may depend on what your fellow students do. Thus, unlike previous lab exercises in which we anticipated a particular outcome, you will only find out whether your evolution program worked well when you test your evolved organism against those of others or against variants you create yourself. Program Steps and Strategies We will ask you to use particular parameter values. You will have to use these values, and submit lab reports (programs and results) with these values to receive credit.

4 Here are parameter values to use to start off. The meanings will be clear when you read below. Population size: 1000 # of bouts: 100 chromosome sizes: 50, 100, 150, 200, 250, or 300 # of individuals with point mutations each generation: 500 # of individuals with size mutations each generation: 100 point mutation range: [0 1400] (real numbers, not only integers) calorie payoff: 120 # generations in Project To facilitate debugging and grading, we ask that you only make functions for tasks where it is explicitly allowed below. Project 1: Standing Variation 1) Create a vector of length 100 to represent the chromosome of one player. Each of the 100 elements in the vector corresponds to a possible investment that the player could play. The element values can be real numbers from 0 to This range of possible plays from 0 to 1400 can never be changed, it is part of the rules or the biology of the organisms. When the player is asked to make a play, a random element is drawn from the chromosome. 2) Create a population of 1,000 players (vectors of chromosomes). The chromosome sizes for the players will vary. When creating the population, choose the chromosome size for each player randomly from the values 50, 100, 150, 200, 250, and 300. We encourage you to use a list to store this population. Unfortunately, vectorization on lists is more complicated than vectorization on vectors and arrays. Later on, if you want to try to vectorize your program, you can think about how you could use an array to store your population. NA and/or rep() might come in handy for that. 3) Later on, you will evolve your population until it reaches fixation. To do so, you will need to figure out when your population reaches fixation. Write a function that will tell you when your population has reached fixation. To test this function, it may be helpful to create a temporary population of 1,000 identical chromosomes and compare your function's output on this population compared to the population from #2. 4) Calculate the performance scores of your 1,000 players by competing each one against a competitor pool of 100 players. This competitor pool should be duplicates of 100 players

5 chosen from your population once per generation and used for all 1,000 players. When you compete a player from your population against a player from the competitor pool you only need to record the payoff to the player from the population. The competitor pool is just there to provide opponents. a. Determine the score of all 1,000 players. The score of the player is the average payoff the player receives from the 100 bouts. Use a value of calorie payoff k=120 calories. If you'd like, you can make functions for: i. taking a chromosome of any size as input and generating a vector of n plays OR generating plays in a vectorized way, and ii. taking two plays and the payoff as input and determining the payoff to each player in a bout OR determining the payoffs in a vectorized way. 5) Once you have determined the scores of your population, use these scores as fitness values for selection. Select for the players with the higher fitness to evolve your best player. Generate an offspring population of the same size as the parent population. To impose one generation of selection, use the sample command and pass in your scores using the prob= argument to the function. Note that the sample command expects anything passed in via prob= to be nonnegative. If some of your scores are negative, you will need to figure out a good way to make these positive while still preserving the relative ranking of different individuals. There are different perfectly acceptable ways to do this. Think carefully about it and decide on one. Before contacting an instructor about how to change scores from negative to positive, you must have thought about it and be able to present a plan. You can make a function for selection and reproduction. 6) Once a new population is created by biased sampling (i.e. selection), this is a new generation. Repeat the process and track the composition of your 1,000 players until you get fixation (see #3). If your populations are not going to fixation after many hundreds of generations, there may be an error in your code. You should examine your program for errors or other possible explanations. Feel free to consult your instructors in this case. Use your program to create an evolved player. Give this player a distinctive name (e.g. superdupergnarlybesteverplayer1) and save its vector for future use. In addition, go to the website and upload your player into the form there using the instructions there. 7) Once you have evolved your player, compete your evolved player against someone else's. You can get other students' players using code on the website. We will complete a round robin and note the outcomes on the board. To compete two evolved players, introduce both players at equal frequencies (500:500 of each) and follow the population for 100 generations. If ties appear, try to think of rules to break a tie. Discuss rules for breaking ties as a class so that we have a uniform criterion. Save the code used for competition and call it the Competition Arena.

6 8) Hand in a lab report describing your program, the outcomes, and how your evolved player fared. Include a copy of your R code use to evolve your player. 9) Optional (but useful for later): Keep track of the composition of your 1,000 players (e.g. how does starting player #3 do over time? how about starting player #565?). How did the eventual winner rank after the first generation? Project 2: New Mutations 1) By completing Project 1, you have imposed selection on a population of players that were generated randomly at the beginning of your program. Their frequencies changed throughout the course of evolution, but the chromosomes themselves did not (i.e. there was no mutation). Copy all your code and functions so that you have the original ones from Project 1 and a new set for Project 2 and clear your environment. In Project 2, you will mutate and change the contents (genotype) of the 1,000 player vectors every generation. 2) Two kinds of mutations are allowed in this simulation: a. A point mutation changes the value of a position in a chromosome. If a position is mutated it is replaced by a random value from 0 to 1,400. Exactly half of your chromosomes will get a point mutation and this consists of one randomly chosen position in the vector being mutated. b. A duplication or deletion changes the size of the chromosome. These work in increments of 50 and the chromosome size is bounded between 50 and 300. So the only valid chromosome sizes are 50, 100, 150, 200, 250, and 300. Choose 100 players in your population to change size each generation and flip a coin (computationally) to determine whether it is a duplication or a deletion. If a chromosome is to change size, you will need to select a contiguous stretch of 50 values at random and either duplicate it by adding it onto the end of your vector or delete it. Write code to mutate the population. You can make functions to: i. take a chromosome of any size as input and mutate a chosen number of positions, and ii. take a chromosome as input and mutate its size. 3) Run the population of 1,000 players through 250 generations of selection and mutagenesis. Mutagenize every generation. 4) Take your best evolved player, give it a name subscripted 2, like superdupergnarlybesteverplayer2 and save its vector. Compete superdupergnarlybesteverplayer1 and superdupergnarlybesteverplayer2 and see which does

7 better. You should use the Competition Arena code you developed in Project 1 because you don't want your organisms mutating while you are a competing them. Project 3: Improve Your Player by Varying the Evolutionary Process 1) If your organisms from Project 2 are better, your program must be doing something correctly. Congratulations! Now tune your program to evolve an even better player. Note that it may not be obvious what is best. A larger population or more generations of selection are always better, but you will need more computation time. Try to balance evolutionary theory and computation time. Some possibilities are: a) Changing population size (greater or less than 1,000)? Natural selection is more efficient with large population sizes. b) Right now, new point mutations are random values between 0 and Perhaps new mutations should depend on what the previous value was. So if you have a play of 400, perhaps your possible new mutated values are +/- 5% (380 to 420). But remember to make sure that the new values are plausible (can't be negative, can't be greater than 1,400). c) Mutagenize more or fewer than 500 players with point mutations or a different number of chromosome size changes? High mutation rates can also do harm. d) Have a mutation rate per value rather than per chromosome. e) Evolve for more than 250 generations? This is always good. f) Add sexual reproduction and recombination between chromosomes. Note that you cannot change k=120 calories, the range of 0 to 1400 for the elements in the chromosome vector, or the allowed chromosome vector length of 50, 100, 150, 200, 250, or 300. k is imposed by the environment, and the chromosome vector sizes are constrained by the biology of the organisms. 2) After you have evolved your improved players in this third project, name them subscripted 3 (superdupergnarlybesteverplayer3) and bring them to the BIEB 143 Olympics. All evolved player3s will be combined into a large population and competed to see who is the final. Note that no mutations are allowed during competition. 3) Hand in a lab report describing your program, the outcomes, and how your evolved player2s and player3s fared. Include a copy of your R code use to evolve your players.

Achieving Desirable Gameplay Objectives by Niched Evolution of Game Parameters

Achieving Desirable Gameplay Objectives by Niched Evolution of Game Parameters Scott Watson, Andrew Vardy, Wolfgang Banzhaf Department of Computer Science Memorial University of Newfoundland St John s.