Wednesday, February 1, 2017

Wednesday, February 1, 2017 Topics for today Encoding game positions Constructing variable-length codes Huffman codes Encoding Game positions Some programs that play two-player games (e.g., tic-tac-toe, mancala, checkers, chess, go use game trees where they represent moves, counter-moves, counter-counter moves and so on. In the following fragment, -1 represents a loss, +1 represents a win. Current position My possible moves Opponent s responses -1 My responses to that +1 The program will explore the tree to a certain depth then choose the best move assuming the opponent plays optimally. Depending on the game, the tree may consist of hundreds of thousands of nodes. Encoding schemes that use the fewest bits to represent a game position are of interest. When encoding a game position we might have a choice between: (1 a fixed-length scheme which always uses the same number of bits per position and (2 a variable-length scheme where the number of bits depends on the number of pieces left in the game. The former might be better in situations where we are storing a sequence of game positions in memory and wish to randomly access a particular one. We can easily compute the address in memory of the k th game representation if they are all the same size. Comp 162 Notes Page 1 of 12 February 1, 2017

Encoding I: tic-tac-toe There are nine squares each of which is either empty or contains X or contains 0 so we have 3 9 possible boards (not all legal. 16384 (2 14 < 19683 (3 9 < 32768 (2 15 so we would need 15 bits. Can you think of a way to use fewer? With symmetries and representing only legal positions? Encoding II: mancala There are 14 pits that can hold stones. There are 48 stones. We would need 6 bits to represent the number of stones in a pit so 14 * 6 = 84 bits altogether. But not every combination is legal (the sum of the numbers cannot exceed 48. Encoding III: checkers Programs that play checkers and chess often store board configurations (hundreds of thousands of them to help determine the best move to make. In addition, we might want to transmit a board configuration to enable a person at the other end of the line to set up a board in a certain way e.g., to solve a Win-in-N-moves puzzle. Q: How many bits are required to transmit a checkers position? Q: How many bits are required to transmit a chess position? Here are a couple of possible encoding methods for checkers: Checkers-1 (fixed length In checkers, only 32 of the 64 squares are used. Each square is either empty or contains a white piece, a white king, a black piece or a black king. Thus there are 5 possibilities => 3 bits. Total number of bits for the board is 32*3 = 96 bits. Checkers-2 (variable length For each one of a player s pieces piece (max 12 pieces for each player use 1 bit to indicate if it is on the board or off the board. If it is on the board then a further 1 bit for the piece type (regular or king and 5 bits for its location (1..32. If all pieces are active (as at the start of a game then this method needs 24*7 = 168 bits but the number of bits required decreases as pieces are taken off the board. When there are fewer than 14 pieces left, it uses fewer bits than Method 1. Comp 162 Notes Page 2 of 12 February 1, 2017

Encoding IV: chess Many schemes have been used for chess. Here is a brief look at 5 of them: Chess-1 (fixed length Chess uses all 64 squares. Encode each of the 64 squares with a 5 bit code. First bit indicates if the square is empty or occupied; second bit indicates if the square is occupied by a black piece or a white piece, the remaining three bits indicate the type of piece (pawn, rook, knight, bishop, queen, king. Coding a board using Method 1 required 320 bits. Chess-2 (fixed length Encode each of the 64 squares with a 4-bit code. First bit is 0/1 indicating if the piece on the square is black or white. The remaining 3 bits indicate the type of piece (if any. For example: 000 empty 001 king 010 queen 011 bishop 100 rook 101 knight 110 pawn Coding a board using Method 2 requires 256 bits. Chess-3 (variable length Encode each piece with its color, type (see above and location, A type 000 means the piece is not on the board. Location requires 6 bits (64 possible squares. Total 10 bits required for each piece on the board. At the start of the game this means 320 bits (same as method 1. Method 3 requires fewer bits than Method 2 if there are fewer than 26 pieces on the board. Chess- 4 (fixed length Encode each piece with 7 bits - 1st bit indicates if the piece is on the board or off the board and the remaining 6 bits give location. For the 32 pieces this requires 7*32 = 224 bits, a little less than Method 2. Comp 162 Notes Page 3 of 12 February 1, 2017

Chess-5 (variable-length A variable length version of method 2. Represent an empty square with code "0" and a non-empty square by a "1" followed by 4 bits - 1 bit black/white, 3 bits piece type. Initial configuration requires 32*1 bits (for the 32 empty squares + 32*5 bits (for the 32 pieces = 192 bits. The number of bits declines with the number of active pieces. So, for checkers and chess, the fixed-length schemes tend to be board-oriented and the variablelength schemes are piece-oriented. Constructing variable-length codes We have seen from the above that codes might be fixed-length or variable-length. (Another example of a variable-length code is the Morse code. Next we look at a method for constructing optimal variable-length codes. (This is not in the book see links to web information on Huffman codes next to the course notes on the course home page Consider the encoding of character data. Rather than using a fixed length scheme such as ASCII (7 bits or EBCDIC (8 bits or Unicode (16 bits we could devise a coding scheme which, like Morse code, uses different length codes for different symbols. If we assign shorter codes to more frequent characters and longer codes to rarer characters this might save storage and/or transmission time. Care is needed that the assignment of codewords to symbols does not lead to ambiguities. For example, if we assigned A = 1 B = 01 C = 001 D = 101 then the string 101 could be interpreted either as "D" or as "AB". So codes must be unambiguous. We would also like them to be instantaneous - no requirement to look ahead one bit when decoding. An example of a code that is unambiguous but not instantaneous is the following: A = 1 B = 10 C = 100 D = 1000 When decoding a string such as 100110001 using this code (it is CADA we need to read one bit ahead in order to know we have reached the end of a codeword. Compare this with A = 1 B = 01 C = 001 D = 0001 and decoding 001100011 Comp 162 Notes Page 4 of 12 February 1, 2017

Huffman codes Huffman codes are * unambiguous * instantaneous * optimal - the technique we look at for devising the codes guarantees that the average weighted codeword length is minimized (more on this later. Here is Huffman s 4-step algorithm for constructing codes. (1 start with items to be encoded. Associate with each one its weight or frequency. (2 while there is more than one item left { combine the two items with lowest weight associate the sum of the two weights with the new item } [If there is a choice to make when selecting the two lowest weights it does not matter which choice we make] (3 label branches of the tree that results. One of the branches from a node should be labeled "1" and the other labeled "0" (4 the code for an item is found by concatenating the bits on the path from the root of the tree to the appropriate node Example Symbol Frequency A 3 B 3 C 2 D 3 E 6 A possible result of Step 2 is the following tree (we made arbitrary choices when choosing between two nodes with the same value. Other trees are possible. Comp 162 Notes Page 5 of 12 February 1, 2017

17 11 6 5 A(3 B(3 C(2 D(3 E(6 Step 3 add labels to branches gives us 1 0 1 0 1 0 1 0 A(3 B(3 C(2 D(3 E(6 Step 4 read out the codewords. Symbol Frequency Codeword A 3 11 B 3 10 C 2 011 D 3 010 E 6 00 Encoding To encode a sequence of symbols replace each symbol by its binary codeword. For example, BEADED is encoded as 10001101000101 Comp 162 Notes Page 6 of 12 February 1, 2017

Decoding To decode a binary string: start at top of tree. Read bits and move down tree left or right according to the labels on the tree branches. Keep reading bits and moving down the tree until you reach a terminal node (leaf. Output the character there and return to the top of the tree. In addition to being unambiguous (guaranteed by the nature of the tree Huffman codes have the property that they have minimal average weighted code length (AWCL. Here is an example of how to calculate this AWCL Example Symbol Frequency Codeword Codeword length Frequency * Codeword length A 3 11 2 6 B 3 10 2 6 C 2 011 3 6 D 3 010 3 9 E 6 00 2 12 AWCL = Σ (freq * codeword length / Σ (frequencies In our example the AWCL is 39/17 = 2.29 A quick way to compute the AWCL is to observe that The numerator is the sum of the numbers on the internal nodes of the tree. (Why? The denominator is the number in the root of the tree. In our example: numerator is (6 + 5 + 11 + 17 = 39, denominator = 17. Huffman s algorithm guarantees that this weighted code length is minimal - there is no other set of codewords that has shorter average weighted code length. In Step 2 we might have choices when combining smallest weights. It turns out that we get the same weighted average code length no matter which choice we make. Here is a different tree derived from our example data (the order of the nodes has been changed to avoid lines crossing in the tree. Comp 162 Notes Page 7 of 12 February 1, 2017

17 11 5 6 B(3 C(2 A(3 D(3 E(6 Adding labels to branches gives us 0 1 0 1 0 1 0 1 B(3 C(2 A(3 D(3 E(6 And codewords Symbol Frequency Codeword A 3 010 B 3 000 C 2 001 D 3 011 E 6 1 The AWCL is (9 + 9 + 6 + 9 + 6 / 17 = 39/17 = 2.29 same as before Comp 162 Notes Page 8 of 12 February 1, 2017

The more non-uniform the frequencies, the larger the difference in codeword lengths. If all the frequencies are the same, the codewords will be much the same length and not a lot of savings result. Here is an example where the frequencies are more skewed. Symbol Frequency Codeword A 2 0000 B 3 0001 C 8 001 D 12 01 E 20 1 The average weighted codeword in this case is 88/45 = 1.96 bits - compare with the 3 bits needed if we use a fixed length code. Here are some questions to think about. Q. Is there a limit on the size of the AWCL? What is the smallest it could be if there are N characters to be encoded? Q. What are the characteristics of the character-frequency table that would result in all characters having codewords of the same length? Q. What are the characteristics of the character-frequency table that would result in no two characters having the same length codeword? Note that there is a problem with a Huffman-encoded text if a bit of the encoded text is corrupted. With ASCII (and similar fixed-length codes, a corrupted bit affects only one character. With Huffman encoding, the rest of the stream may be interpreted incorrectly or become unreadable. For example, take our earlier encoding of BEADED (end of page 6 10001101000101 If we change it to 10011101000101 it decodes to BCBBEB If we change it to 10001101010101 it decodes to BEADBB In these two cases we would be partway through decoding a symbol when we run out of input, a sure sign something has gone wrong. This may not always happen as in the following If we change it to 00011101000101 it decodes to EEADED with no indication of error. Comp 162 Notes Page 9 of 12 February 1, 2017

More Huffman examples Suppose the symbols and their weights are as follows A B C D E 80 40 40 20 20 When applying Huffman s algorithm to build the tree we will sometimes have choices as to which nodes to combine. Each of the following three trees is a possible result. 200 200 200 120 120 80 80 80 40 120 40 40 80 40 40 20 20 80 40 40 20 20 80 40 40 20 20 A B C D E A B C D E A B C D E With corresponding codewords (we don t care about actual 1 s and 0 s A * A ** A * B *** B ** B ** C *** C ** C *** D *** D *** D **** E *** E *** E **** But the AWCL = (sum of internal numbers/ number at top = 440/200 = 2.2 in all cases. Reading: Review some of the links to Huffman codes. Start to look at Chapter 4. We will move fairly quickly through Chapter 4 in order to get to the Assembly language level in Chapter 5. Pep/9 programming assignments in this class will be in Assembly language not in hexadecimal. Comp 162 Notes Page 10 of 12 February 1, 2017

Review Questions 1. What is the smallest number of bits required to record the status of an inning in baseball? Data to be stored includes number of outs, ball/strike count and runners on base. State any assumptions you make. 2. If one of the items to be encoded with a Huffman code has frequency/probability of zero, will it be assigned a code? 3. If only one of the items to be encoded with a Huffman code has a non-zero frequency/probability, what can you say about the code assigned to this item. 4. If there are 2 N items to be encoded with a Huffman code and all have the same frequency/probability, what can you say about the codes assigned? 5. How many other tic-tac-toe boards are equivalent to X.. X O. O..? 6. You enter a classroom and see the Huffman code tree below on the board. The frequencies of the symbols have been erased. Add frequencies to the tree consistent with its shape. A B C D E F Comp 162 Notes Page 11 of 12 February 1, 2017

Review Answers 1. Nine bits could be used as follows: 2. Yes. 3 record, for each of 1 st, 2 nd and 3 rd base, whether there is a runner on base. 2 record the number of outs 0, 1, 2 2 record the number of strikes 0, 1, 2 2 record the number of balls 0,1,2,3 Assumes that if the batter walks, the count is reset so no need to store Ball 4. 3. It will be 1-bit long. 4. They will all be N bits long 5. At least these 10 O X X O. O.. O.. X O. O. O.. O X. O.. O X X O. O.... X X X O O. O X.. X O O X. X.. X.. O X. X. X.. X O. X.. X O O X. X.... O O O X X. X O.. 6. Many answers are possible. Below is one. Check your answer by building the tree from your frequencies and see if it can possible match the given shape. A B C D E F 10 10 15 30 10 10 Comp 162 Notes Page 12 of 12 February 1, 2017