Lossless Abstraction of Imperfect Information Games

Transcription

1 Lossless Abstracton of Imperfect Informaton Games ANDREW GILPIN AND TUOMAS SANDHOLM Carnege Mellon Unversty, Pttsburgh, Pennsylvana Abstract. Fndng an equlbrum of an extensve form game of mperfect nformaton s a fundamental problem n computatonal game theory, but current technques do not scale to large games. To address ths, we ntroduce the ordered game somorphsm and the related ordered game somorphc abstracton transformaton. For a mult-player sequental game of mperfect nformaton wth observable actons and an ordered sgnal space, we prove that any Nash equlbrum n an abstracted smaller game, obtaned by one or more applcatons of the transformaton, can be easly converted nto a Nash equlbrum n the orgnal game. We present an algorthm, GameShrnk, for abstractng the game usng our somorphsm exhaustvely. Its complexty s Õn 2, where n s the number of nodes n a structure we call the sgnal tree. It s no larger than the game tree, and on nontrval games t s drastcally smaller, so GameShrnk has tme and space complexty sublnear n the sze of the game tree. Usng GameShrnk, we fnd an equlbrum to a poker game wth 3.1 bllon nodes over four orders of magntude more than n the largest poker game solved prevously. To address even larger games, we ntroduce approxmaton methods that do not preserve equlbrum, but nevertheless yeld ex post provably close-to-optmal strateges. Categores and Subject Descrptors: F.2.0 [Analyss of Algorthms and Problem Complexty]: General; I.2.0 [Artfcal Intellgence]: General; J.4 [Socal and Behavoral Scences]: Economcs General Terms: Algorthms, Economcs, Theory Addtonal Key Words and Phrases: Automated abstracton, computer poker, equlbrum fndng, game theory, sequental games of mperfect nformaton ACM Reference Format: Glpn, A. and Sandholm, T Lossless abstracton of mperfect nformaton games. J. ACM 54, 5, Artcle 25 October 2007, 30 pages. DOI = / / Ths materal s based on work supported by the Natonal Scence Foundaton NSF under ITR grants IIS and IIS , and Sloan Fellowshp. A short verson of ths artcle appeared n Proceedngs of the ACM Conference on Electronc Commerce EC Ann Arbor, MI, June, ACM, New York, Authors address: Carnege Mellon Unversty, School of Computer Scence, 5000 Forbes Avenue, Pttsburgh, PA , e-mal: {glpn; sandholm}@cs.cmu.edu. Permsson to make dgtal or hard copes of part or all of ths work for personal or classroom use s granted wthout fee provded that copes are not made or dstrbuted for proft or drect commercal advantage and that copes show ths notce on the frst page or ntal screen of a dsplay along wth the full ctaton. Copyrghts for components of ths work owned by others than ACM must be honored. Abstractng wth credt s permtted. To copy otherwse, to republsh, to post on servers, to redstrbute to lsts, or to use any component of ths work n other works requres pror specfc permsson and/or a fee. Permssons may be requested from Publcatons Dept., ACM, Inc., 2 Penn Plaza, Sute 701, New York, NY USA, fax , or permssons@acm.org. C 2007 ACM /2007/10-ART25 $5.00 DOI / / Journal of the ACM, Vol. 54, No. 5, Artcle 25, Publcaton date: October 2007.

2 25:2 A. GILPIN AND T. SANDHOLM 1. Introducton In envronments wth more than one agent, an agent s outcome s generally affected by the actons of the other agents. Consequently, the optmal strategy of one agent can depend on the others. Game theory provdes a normatve framework for analyzng such strategc stuatons. In partcular, t provdes soluton concepts that defne what ratonal behavor s n such settngs. The most famous and mportant soluton concept s that of Nash equlbrum [Nash 1950]. It s a strategy profle one strategy for each agent n whch no agent has ncentve to devate to a dfferent strategy. However, for the concept to be operatonal, we need algorthmc technques for fndng an equlbrum. Games can be classfed as ether games of perfect nformaton or mperfect nformaton. Chess and Go are examples of the former, and, untl recently, most game playng research has been on games of ths type. To compute an optmal strategy n a perfect nformaton game, an agent traverses the game tree and evaluates ndvdual nodes. If the agent s able to traverse the entre game tree, she smply computes an optmal strategy from the bottom-up, usng the prncple of backward nducton. 1 In computer scence terms, ths s done usng mnmax search often n conjuncton wth α-β-prunng to reduce the search tree sze and thus enhance speed. Mnmax search runs n lnear tme n the sze of the game tree. 2 The dfferentatng feature of games of mperfect nformaton, such as poker, s that they are not fully observable: when t s an agent s turn to move, she does not have access to all of the nformaton about the world. In such games, the decson of what to do at a pont n tme cannot generally be optmally made wthout consderng decsons at all other ponts n tme ncludng ones on other paths of play because those other decsons affect the probabltes of beng at dfferent states at the current pont n tme. Thus, the algorthms for perfect nformaton games do not solve games of mperfect nformaton. For sequental games wth mperfect nformaton, one could try to fnd an equlbrum usng the normal matrx form, where every contngency plan of the agent s a pure strategy for the agent. 3 Unfortunately even f equvalent strateges are 1 Ths actually yelds a soluton that satsfes not only the Nash equlbrum soluton concept, but a stronger soluton concept called subgame perfect Nash equlbrum [Selten 1965]. 2 Ths type of algorthm stll does not scale to huge trees such as n chess or Go, but effectve game-playng agents can be developed even then by evaluatng ntermedate nodes usng a heurstc evaluaton and then treatng those nodes as leaves. 3 There has been sgnfcant recent work on Nash equlbrum fndng for normal matrx form games. An ε-equlbrum n a normal form game wth any constant number of agents can be constructed n quas-polynomal tme [Lpton et al. 2003], but fndng an exact equlbrum s PPAD-complete even n a 2-player game [Chen and Deng 2006]. The most prevalent algorthm for fndng an equlbrum n a 2-agent game s Lemke-Howson [Lemke and Howson 1964], but t takes exponentally many steps n the worst case [Savan and von Stengel 2004]. For a survey of equlbrum computaton n 2-player games, see von Stengel [2002]. Equlbrum-fndng algorthms that enumerate supports.e., sets of pure strateges that are played wth postve probablty have been shown effcent on many games [Porter et al. 2004], and effcent mxed nteger programmng algorthms that search n the space of supports have been developed [Sandholm et al. 2005]. For more than two players, many algorthms have been proposed, but they currently only scale to very small games [Govndan and Wlson 2003; McKelvey and McLennan 1996; Porter et al. 2004]. Progress has also been made on algorthms for fndng equlbra n restrcted and/or structured games e.g., Papadmtrou and Roughgarden [2005], Bhat and Leyton-Brown [2004], Leyton-Brown and Tennenholtz [2003], Blum et al. [2003], and Sngh et al. [2004]. Journal of the ACM, Vol. 54, No. 5, Artcle 25, Publcaton date: October 2007.

3 Lossless Abstracton of Imperfect Informaton Games 25:3 replaced by a sngle strategy [Kuhn 1950a] ths representaton s generally exponental n the sze of the game tree [von Stengel 1996]. By observng that one needs to consder only sequences of moves rather than pure strateges [Romanovsk 1962; Selten 1988; Koller and Megddo 1992; von Stengel 1996], one arrves at a more compact representaton, the sequence form, whch s lnear n the sze of the game tree. 4 For 2-player games, there s a polynomal-szed n the sze of the game tree lnear programmng formulaton lnear complementarty n the non-zero-sum case based on the sequence form such that strateges for players 1 and 2 correspond to prmal and dual varables. Thus, the equlbra of reasonable-szed 2-player games can be computed usng ths method [von Stengel 1996; Koller et al. 1996; Koller and Pfeffer 1997]. 5 However, ths approach stll yelds enormous unsolvable optmzaton problems for many real-world games, such as poker OUR APPROACH. In ths artcle, we take a dfferent approach to tacklng the dffcult problem of equlbrum computaton. Instead of developng an equlbrumfndng method per se, we nstead develop a methodology for automatcally abstractng games n such a way that any equlbrum n the smaller abstracted game corresponds drectly to an equlbrum n the orgnal game. Thus, by computng an equlbrum n the smaller game usng any avalable equlbrum-fndng algorthm, we are able to construct an equlbrum n the orgnal game. The motvaton s that an equlbrum for the smaller game can be computed drastcally faster than for the orgnal game. To ths end, we ntroduce games wth ordered sgnals Secton 2, a broad class of games that has enough structure whch we can explot for abstracton purposes. Instead of operatng drectly on the game tree somethng we found to be techncally challengng, we nstead ntroduce the use of nformaton flters Secton 2.2, whch coarsen the nformaton each player receves. They are used n our analyss and abstracton algorthm. By operatng only n the space of flters, we are able to keep the strategc structure of the game ntact, whle abstractng out detals of the game n a way that s lossless from the perspectve of equlbrum fndng. We ntroduce the ordered game somorphsm to descrbe strategcally symmetrc stuatons and the ordered game somorphc abstracton transformaton to take advantange of such symmetres Secton 3. As our man equlbrum result we have the followng: THEOREM 3.4. Let Ɣ be a game wth ordered sgnals, and let F be an nformaton flter for Ɣ. Let F be an nformaton flter constructed from F by one applcaton of the ordered game somorphc abstracton transformaton, and let σ be a Nash equlbrum strategy profle of the nduced game Ɣ F.e., the game Ɣ usng the flter F. If σ s constructed by usng the correspondng strateges of σ, then σ s a Nash equlbrum of Ɣ F. 4 There were also early technques that captalzed n dfferent ways on the fact that n many games the vast majorty of pure strateges are not played n equlbrum [Wlson 1972; Koller and Megddo 1996]. 5 Recently, ths approach was extended to handle computng sequental equlbra [Kreps and Wlson 1982] as well [Mltersen and Sørensen 2006]. Journal of the ACM, Vol. 54, No. 5, Artcle 25, Publcaton date: October 2007.

4 25:4 A. GILPIN AND T. SANDHOLM The proof of the theorem uses an equvalent characterzaton of Nash equlbra: σ s a Nash equlbrum f and only f there exst belefs μ players belefs about unknown nformaton at all ponts of the game reachable by σ such that σ s sequentally ratonal.e., a best response gven μ, where μ s updated usng Bayes rule. We can then use the fact that σ s a Nash equlbrum to show that σ s a Nash equlbrum consderng only local propertes of the game. We also gve an algorthm, GameShrnk, for abstractng the game usng our somorphsm exhaustvely Secton 4. Its complexty s Õn 2, where n s the number of nodes n a structure we call the sgnal tree. It s no larger than the game tree, and on nontrval games t s drastcally smaller, so GameShrnk has tme and space complexty sublnear n the sze of the game tree. We present several algorthmc and data structure related speed mprovements Secton 4.1, and we demonstrate how a smple modfcaton to our algorthm yelds an approxmaton algorthm Secton APPLICATIONS. Sequental games of mperfect nformaton are ubqutous, for example n negotaton and n auctons. Often aspects of a player s knowledge are not pertnent for decdng what acton the player should take at a gven pont n the game. On the trval end, some aspects of a player s knowledge are never pertnent e.g., whether t s ranng or not has no bearng on the bddng strategy n an art aucton, and such aspects can be completely left out of the model specfcaton. However, more generally, some aspects can be pertnent n certan states of the game whle they are not pertnent n other states, and thus cannot be left out of the model completely. Furthermore, t may be hghly nonobvous whch aspects are pertnent n whch states of the game. Our algorthm automatcally dscovers whch aspects are rrelevant n dfferent states, and elmnates those aspects of the game, resultng n a more compact, equvalent game representaton. One broad applcaton area that has ths property s sequental negotaton potentally over multple ssues. Another broad applcaton area s sequental auctons potentally over multple goods. For example, n those states of a 1-object aucton where bdder A can nfer that hs valuaton s greater than that of bdder B, bdder A can gnore all hs other nformaton about B s sgnals, although that nformaton would be relevant for nferrng B s exact valuaton. Furthermore, n some states of the aucton, a bdder mght not care whch exact other bdders have whch valuatons, but cares about whch valuatons are held by the other bdders n aggregate gnorng ther denttes. Many open-cry sequental aucton and negotaton mechansms fall wthn the game model studed n ths artcle specfed n detal later, as do certan other games n electronc commerce, such as sequences of take-t-or-leave-t offers [Sandholm and Glpn 2006]. Our technques are n no way specfc to an applcaton. The man experment that we present n ths artcle s on a recreatonal game. We chose a partcular poker game as the benchmark problem because t yelds an extremely complcated and enormous game tree, t s a game of mperfect nformaton, t s fully specfed as a game and the data s avalable, and t has been posted as a challenge problem by others [Sh and Lttman 2002] to our knowledge no such challenge problem nstances have been proposed for electronc commerce applcatons that requre solvng sequental games RHODE ISLAND HOLD EM POKER. Poker s an enormously popular card game played around the world. The 2005 World Seres of Poker had over Journal of the ACM, Vol. 54, No. 5, Artcle 25, Publcaton date: October 2007.

5 Lossless Abstracton of Imperfect Informaton Games 25:5 $103 mllon dollars n total prze money, ncludng $56 mllon for the man event. Increasngly, poker players compete n onlne casnos, and televson statons regularly broadcast poker tournaments. Poker has been dentfed as an mportant research area n AI due to the uncertanty stemmng from opponents cards, opponents future actons, and chance moves, among other reasons [Bllngs et al. 2002]. Almost snce the feld s foundng, game theory has been used to analyze dfferent aspects of poker [Kuhn 1950b; Nash and Shapley 1950; Bellman and Blackwell 1949; von Neumann and Morgenstern 1947, pp ]. However, ths work was lmted to tny games that could be solved by hand. More recently, AI researchers have been applyng the computatonal power of modern hardware to computng game theory-based strateges for larger games. Koller and Pfeffer determned solutons to poker games wth up to 140,000 nodes usng the sequence form and lnear programmng [Koller and Pfeffer 1997]. Large-scale approxmatons have been developed [Bllngs et al. 2003], but those methods do not provde any guarantees about the performance of the computed strateges. Furthermore, the approxmatons were desgned manually by a human expert. Our approach yelds an automated abstracton mechansm along wth theoretcal guarantees on the strateges performance. Rhode Island Hold em was nvented as a testbed for computatonal game playng [Sh and Lttman 2002]. It was desgned so that t was smlar n style to Texas Hold em, yet not so large that devsng reasonably ntellgent strateges would be mpossble. The rules of Rhode Island Hold em are gven n Secton 2.1. That secton also shows how Rhode Island Hold em can be modeled as a game wth ordered sgnals, that s, t fts n our model. We appled the technques developed n ths paper to fnd an exact mnmax soluton to Rhode Island Hold em, whch has a game tree exceedng 3.1 bllon nodes. Applyng the sequence form to Rhode Island Hold em drectly wthout abstracton yelds a lnear program wth 91,224,226 rows, and the same number of columns. Ths s much too large for current lnear programmng algorthms to handle. We used our GameShrnk algorthm to reduce ths through lossless abstracton, and t yelded a lnear program wth 1,237,238 rows and columns wth 50,428,638 non-zero coeffcents. We then appled terated elmnaton of domnated strateges, whch further reduced ths to 1,190,443 rows and 1,181,084 columns. Applyng terated elmnaton of domnated strateges wthout GameShrnk yelded 89,471,986 rows and 89,121,538 columns, whch stll would have been prohbtvely large to solve. GameShrnk requred less than one second to perform the shrnkng.e., to compute all of the ordered game somorphc abstracton transformatons. Usng a 1.65GHz IBM eserver p5 570 wth 64 ggabytes of RAM the lnear program solver actually needed 25 ggabytes, we solved t n 7 days and 17 hours usng the nteror-pont barrer method of CPLEX verson We demonstrated our optmal Rhode Island Hold em poker player at the AAAI-05 conference [Glpn and Sandholm 2005], and t s avalable for play on-lne at Whle others have worked on computer programs for playng Rhode Island Hold em [Sh and Lttman 2002], no optmal strategy has been found before. Ths s the largest poker game solved to date by over four orders of magntude. Journal of the ACM, Vol. 54, No. 5, Artcle 25, Publcaton date: October 2007.

6 25:6 A. GILPIN AND T. SANDHOLM 2. Games wth Ordered Sgnals We work wth a slghtly restrcted class of games, as compared to the full generalty of the extensve form. 6 Ths class, whch we call games wth ordered sgnals, s hghly structured, but stll general enough to capture a wde range of strategc stuatons. A game wth ordered sgnals conssts of a fnte number of rounds. Wthn a round, the players play a game on a drected tree the tree can be dfferent n dfferent rounds. The only uncertanty players face stems from prvate sgnals the other players have receved and from the unknown future sgnals. In other words, players observe each others actons, but potentally not nature s actons. In each round, there can be publc sgnals announced to all players and prvate sgnals confdentally communcated to ndvdual players. For smplcty, we assume as s the case n most recreatonal games that wthn each round, the number of prvate sgnals receved s the same across players ths could qute lkely be relaxed. We also assume that the legal actons that a player has are ndependent of the sgnals receved. For example, n poker, the legal bettng actons are ndependent of the cards receved. Fnally, the strongest assumpton s that there s a partal orderng over sets of sgnals, and the payoffs are ncreasng not necessarly strctly n these sgnals. For example, n poker, ths partal orderng corresponds exactly to the rankng of card hands. Defnton 2.1. A game wth ordered sgnals s a tuple Ɣ = I, G, L,,κ, γ, p,,ω,u where: 1 I ={1,...,n} s a fnte set of players. 2 G = G 1,...,G r, G j = V j, E j, s a fnte collecton of fnte drected trees wth nodes V j and edges E j. Let Z j denote the leaf nodes of G j and let N j v denote the outgong neghbors of v V j. G j s the stage game for round j. 3 L = L 1,...,L r, L j : V j \ Z j I ndcates whch player acts chooses an outgong edge at each nternal node n round j. 4 s a fnte set of sgnals. 5 κ = κ 1,...,κ r and γ = γ 1,...,γ r are vectors of nonnegatve ntegers, where κ j and γ j denote the number of publc and prvate sgnals per player, respectvely, revealed n round j. Each sgnal θ may only be revealed once, and n each round every player receves the same number of prvate sgnals, so we requre r j=1 κ j + nγ j. The publc nformaton revealed n round j s α j κ j and the publc nformaton revealed n all rounds up through round j s α j = α 1,...,α j. The prvate nformaton revealed to player I n round j s β j γ j and the prvate nformaton revaled to player I n all rounds up through round j s β j = β 1,...,βj. We also wrte β j = β j 1,..., β n j to represent all prvate nformaton up through round j, and β j, β j = β j 1,..., β j 1, β j, β j +1,..., β n j s β j wth β j replaced wth β j. The total nformaton revealed up through round j, α j, β j, s sad to be legal f no sgnals are repeated. 6 For readers unfamlar wth extensve form games, we provde a complete defnton n Appendx A. Journal of the ACM, Vol. 54, No. 5, Artcle 25, Publcaton date: October 2007.

7 Lossless Abstracton of Imperfect Informaton Games 25:7 6 p s a probablty dstrbuton over, wth pθ > 0 for all θ. Sgnals are drawn from accordng to p wthout replacement, so f X s the set of sgnals already revealed, then { px px X = y / X py f x / X 0 f x X. 7 s a partal orderng of subsets of and s defned for at least those pars requred by u. 8 ω : r j=1 Z j {over, contnue} s a mappng of termnal nodes wthn a stage game to one of two values: over, n whch case the game ends, or contnue, n whch case the game contnues to the next round. Clearly, we requre ωz = over for all z Z r. Note that ω s ndependent of the sgnals. Let ωover j ={z Z j ωz = over} and ω j cont ={z Z j ωz = contnue}. 9 u = u 1,...,u r, u j : j 1 k=1 ωk cont ω over j j k=1 κk n j =1 k=1 γ k R n s a utlty functon such that for every j, 1 j r, for every I, and for every z j 1 k=1 ωk cont ωover, j at least one of the followng two condtons holds: a Utlty s sgnal ndependent: u j z,ϑ = u j z,ϑ for all legal ϑ, ϑ j k=1 κk n j =1 k=1 γ k. b s defned for all legal sgnals α j, β j and α j, β j through round j and a player s utlty s ncreasng n her prvate sgnals, everythng else equal: α j, β j α j, β j = u z, α j, β j, β j u z, α j, β j, β j. We wll use the term game wth ordered sgnals and the term ordered game nterchangeably RHODE ISLAND HOLD EM MODELED AS AN ORDERED GAME. In ths secton, we descrbe how Rhode Island Hold em can be defned as an ordered game n accordance wth Defnton 2.1. Frst, we descrbe the rules of Rhode Island Hold em. 1 Each player pays an ante of 5 chps whch s added to the pot. Both players ntally receve a sngle card, face down; these are known as the hole cards. 2 After recevng the hole cards, the players partcpate n one bettng round. Each player may check not placng any money n the pot and passng or bet placng 10 chps nto the pot f no bets have been placed. If a bet has been placed, then the player may fold thus forfetng the game along wth any money they have put nto the pot, call addng chps to the pot equal to the last player s bet, or rase callng the current bet and makng an addtonal bet. In Rhode Island Hold em, the players are lmted to three bets each per bettng round. A rase equals two bets. In the frst bettng round, the bets are equal to 10 chps. 3 After the frst bettng round, a communty card s dealt face up. Ths s called the flop card. Another bettng round take places at ths pont, wth bets equal to 20 chps. 4 Followng the second bettng round, another communty card s dealt face up. Ths s called the turn card. A fnal bettng round takes place at ths pont, wth bets agan equal to 20 chps. Journal of the ACM, Vol. 54, No. 5, Artcle 25, Publcaton date: October 2007.

8 25:8 A. GILPIN AND T. SANDHOLM TABLE I. RANKING OF THREE-CARDPOKER HANDS, FROM HIGHEST TO LOWEST Hand Prob. Descrpton Example Straght flush cards w/ consecutve rank & same sut K, Q, J Three of a knd cards of the same rank Q, Q, Q Straght cards w/ consecutve rank 3, 4, 5 Flush cards of the same sut 2, 5, 8 Par cards of the same rank 2, 2, 3 Hgh card None of the above J, 9, 2 FIG. 1. Stage game G RI, player label L, and game-endng nodes ω for Rhode Island Hold em. The acton labels denote whch acton the player s takng: k check, b bet, f fold, c call, and r rase. Lower case letters ndcate player 1 actons and upper case letters ndcate player 2 actons. 5 If nether player folds, then the showdown takes place. Both players turn over ther cards. The player who has the best 3-card poker hand takes the pot. In the event of a draw, the pot s splt evenly. Hands n 3-card poker games are ranked slghtly dfferently than 5-card poker hands. The man dfferences are that the order of flushes and straghts are reversed, and a three of a knd s better than straghts or flushes. Table I descrbes the rankngs. Wthn ranks, tes are broken by by orderng hands accordng to the rank of cards that make up the hand. If players are stll ted after applyng ths crteron, kckers are used to determne the wnner. A kcker s a card that s not used to make up the hand. For example, f player 1 has a par of eghts and a fve, and player 2 has a par of eghts and a sx, player 2 wns. To make the defnton of ordered games concrete, here we defne each of the components of the tuple Ɣ = I, G, L,,κ,γ,p,,ω,u for Rhode Island Hold em. There are two players so I ={1, 2}. There are three rounds, and the stage game s the same n each round so we have G = G RI, G RI, G RI where G RI s gven n Fgure 1, whch also specfes the player label L. s the standard deck of 52 cards. The communty cards are dealt n the second and thrd rounds, so κ = 0, 1, 1. Each player receves a snce face down card n the frst round only, so γ = 1, 0, 0. p s the unform dstrbuton over. s defned for three card hands and s defned usng the rankng gven n Table I. The game-endng nodes ω are denoted n Fgure 1 by ω. u s defned Journal of the ACM, Vol. 54, No. 5, Artcle 25, Publcaton date: October 2007.

9 Lossless Abstracton of Imperfect Informaton Games 25:9 as n the above descrpton; t s easy to verfy that t satsfes the necessary condtons INFORMATION FILTERS. In ths secton, we defne an nformaton flter for ordered games. Instead of completely revealng a sgnal ether publc or prvate to a player, the sgnal frst passes through ths flter, whch outputs a coarsened sgnal to the player. By varyng the flter appled to a game, we are able to obtan a wde varety of games whle keepng the underlyng acton space of the game ntact. We wll use ths when desgnng our abstracton technques. Formally, an nformaton flter s as follows: Defnton 2.2. Let Ɣ = I, G, L,,κ,γ,p,,ω,u be an ordered game. Let S j j k=1 κk j k=1 γ k be the set of legal sgnals.e., no repeated sgnals for one player through round j. Annformaton flter for Ɣ s a collecton F = F 1,...,F r where each F j s a functon F j : S j 2 S j such that each of the followng condtons hold: 1 Truthfulness α j, β j F j α j, β j for all legal α j, β j. 2 Independence The range of F j s a partton of S j. 3 Informaton preservaton If two values of a sgnal are dstngushable n round k, then they are dstngushable for each round j > k. Let m j = j l=1 κl + γ l. We requre that for all legal θ 1,...,θ m k,...,θ m j and θ 1,...,θ,...,θ : m k m j θ 1,...,θ m / F k θ k 1,...,θ m k = θ 1,...,θ m,...,θ k m / F j θ j 1,...,θ m k,...,θ m j. A game wth ordered sgnals Ɣ and an nformaton flter F for Ɣ defnes a new game Ɣ F. We refer to such games as fltered ordered games. We are left wth the orgnal game f we use the dentty flter F j α j, β j ={ α j, β j }. Wehavethe followng smple but mportant result: PROPOSITION 2.3. A fltered ordered game s an extensve form game satsfyng perfect recall. For the unfamlar reader, the defnton of games wth perfect recall s gven n Appendx A. A smple proof proceeds by constructng an extensve form game drectly from the ordered game, and showng that t satsfes perfect recall. In determnng the payoffs n a game wth fltered sgnals, we take the average over all real sgnals n the fltered class, weghted by the probablty of each real sgnal occurrng STRATEGIES AND NASH EQUILIBRIUM. We are now ready to defne behavor strateges n the context of fltered ordered games. Defnton 2.4. A behavor strategy for player n round j of Ɣ = I, G, L,,κ,γ,p,,ω,u wth nformaton flter F s a probablty dstrbuton over possble actons, and s defned for each player, each round j, and each v V j \ Z j for L j v = : σ j j 1,v : ω k cont Range F j { w V j v, w E j}. k=1 Journal of the ACM, Vol. 54, No. 5, Artcle 25, Publcaton date: October 2007.

10 25:10 A. GILPIN AND T. SANDHOLM FIG.2. GameShrnk appled to a tny two-person four-card two Jacks and two Kngs poker game. Next to each game tree s the range of the nformaton flter F. Dotted lnes denote nformaton sets, whch are labeled by the controllng player. Open crcles are chance nodes wth the ndcated transton probabltes. The root node s the chance node for player 1 s card, and the next level s for player 2 s card. The payment from player 2 to player 1 s gven below each leaf. In ths example, the algorthm reduces the game tree from 113 nodes to 39 nodes. X s the set of probablty dstrbutons over a fnte set X. A behavor strategy for player n round j s σ j = σ j,v 1,...,σ j,v m for each v k V j \ Z j where L j v k =. A behavor strategy for player n Ɣ s σ = σ 1,...,σr. A strategy profle s σ = σ 1,...,σ n. A strategy profle wth σ replaced by σ s σ,σ = σ 1,...,σ 1,σ,σ +1,...,σ n. By an abuse of notaton, we wll say player receves an expected payoff of u σ when all players are playng the strategy profle σ. Strategy σ s sad to be player s best response to σ f for all other strateges σ for player we have u σ,σ u σ,σ. σ s a Nash equlbrum f, for every player, σ s a best response for σ. A Nash equlbrum always exsts n fnte extensve form games [Nash 1950], and one exsts n behavor strateges for games wth perfect recall [Kuhn 1953]. Usng these observatons, we have the followng corollary to Proposton 2.3: COROLLARY 2.5. behavor strateges. For any fltered ordered game, a Nash equlbrum exsts n 3. Equlbrum-Preservng Abstractons In ths secton, we present our man technque for reducng the sze of games. We begn by defnng a fltered sgnal tree whch represents all of the chance moves n the game. The bold edges.e., the frst two levels of the tree n the game trees n Fgure 2 correspond to the fltered sgnal trees n each game. Journal of the ACM, Vol. 54, No. 5, Artcle 25, Publcaton date: October 2007.

11 Lossless Abstracton of Imperfect Informaton Games 25:11 Defnton 3.1. Assocated wth every ordered game Ɣ = I, G, L,,κ, γ, p,,ω,u and nformaton flter F s a fltered sgnal tree, a drected tree n whch each node corresponds to some revealed fltered sgnals and edges correspond to revealng specfc fltered sgnals. The nodes n the fltered sgnal tree represent the set of all possble revealed fltered sgnals publc and prvate at some pont n tme. The fltered publc sgnals revealed n round j correspond to the nodes n the κ j levels begnnng at level j 1 k=1 κ k + nγ k and the prvate sgnals revealed n round j correspond to the nodes n the nγ j levels begnnng at level j k=1 κk + j 1 k=1 nγ k. We denote chldren of a node x as Nx. In addton, we assocate weghts wth the edges correspondng to the probablty of the partcular edge beng chosen gven that ts parent was reached. In many games, there are certan stuatons n the game that can be thought of as beng strategcally equvalent to other stuatons n the game. By meldng these stuatons together, t s possble to arrve at a strategcally equvalent smaller game. The next two defntons formalze ths noton va the ntroducton of the ordered game somorphc relaton and the ordered game somorphc abstracton transformaton. Defnton 3.2. Two subtrees begnnng at nternal nodes x and y of a fltered sgnal tree are ordered game somorphc f x and y have the same parent and there s a bjecton f : Nx Ny, such that for w Nx and v Ny, v = f w mples the weghts on the edges x, w and y, v are the same and the subtrees begnnng at w and v are ordered game somorphc. Two leaves correspondng to fltered sgnals ϑ and ϑ up through round r are ordered game somorphc f for all z r 1 j=1 ω j cont ωover r, ur z,ϑ = u r z,ϑ. Defnton 3.3. Let Ɣ = I, G, L,,κ,γ,p,,ω,u be an ordered game and let F be an nformaton flter for Ɣ. Let ϑ and ϑ be two nformaton structures where the subtrees n the nduced fltered sgnal tree correspondng to the nodes ϑ and ϑ are ordered game somorphc, and ϑ and ϑ are at ether level j 1 k=1 κk + nγ k or j k=1 κk + j 1 k=1 nγ k for some round j. The ordered game somorphc abstracton transformaton s gven by creatng a new nformaton flter F : { F F j α j, β j j α j, β j f α j, β j / ϑ ϑ = ϑ ϑ f α j, β j ϑ ϑ. Fgure 2 shows the ordered game somorphc abstracton transformaton appled twce to a tny poker game. Theorem 3.4, our man equlbrum result, shows how the ordered game somorphc abstracton transformaton can be used to compute equlbra faster. THEOREM 3.4. Let Ɣ = I, G, L,,κ,γ,p,,ω,u be an ordered game and F be an nformaton flter for Ɣ. Let F be an nformaton flter constructed from F by one applcaton of the ordered game somorphc abstracton transformaton. Let σ be a Nash equlbrum of the nduced game Ɣ F. If we take σ j,v z, F j α j, β j = σ j,v z, F j α j, β j, σ s a Nash equlbrum of Ɣ F. PROOF. For an extensve form game, a belef system μ assgns a probablty to every decson node x such that x h μx = 1 for every nformaton set h. A Journal of the ACM, Vol. 54, No. 5, Artcle 25, Publcaton date: October 2007.

12 25:12 A. GILPIN AND T. SANDHOLM strategy profle σ s sequentally ratonal at h gven belef system μ f u σ,σ h,μ u τ,σ h,μ for all other strateges τ, where s the player who controls h. A basc result [Mas-Colell et al. 1995, Proposton 9.C.1] characterzng Nash equlbra dctates that σ s a Nash equlbrum f and only f there s a belef system μ such that for every nformaton set h wth Prh σ > 0, the followng two condtons hold: C1 σ s sequentally ratonal at h gven μ; and C2 μx = Prx σ for all x h. Snce Prh σ σ s a Nash equlbrum of Ɣ, there exsts such a belef system μ for Ɣ F. Usng μ, we wll construct a belef system μ for Ɣ and show that condtons C1 and C2 hold, thus supportng σ as a Nash equlbrum. Fx some player I. Each of s nformaton sets n some round j corresponds to fltered sgnals F j α j, β j, hstory n the frst j 1 rounds z 1,...,z j 1 j 1 k=1 ωk cont, and hstory so far n round j, v V j \ Z j. Let z = z 1,...,z j 1, v represent all of the player actons leadng to ths nformaton set. Thus, we can unquely specfy ths nformaton set usng the nformaton F j α j, β j, z. Each node n an nformaton set corresponds to the possble prvate sgnals the other players have receved. Denote by β some legal F j α j, β j 1,...,F j α j, β j 1, F j α j, β j +1,...,F j α j, β n j. In other words, there exsts α j, β j 1,..., β n j such that α j, β j F j α j, β j, α j, β j k F j α j, β j k for k, and no sgnals are repeated. Usng such a set of sgnals α j, β j 1,..., β n j, let ˆβ denote F j α j, β j 1,..., F j α j, β j 1, F j α j, β j +1,...,F j α j, β n j. We wll abuse notaton and wrte F j ˆβ = ˆβ. We can now compute μ drectly from μ : μ ˆβ F j α j, β j, z μ ˆβ F j α j, β j f F j α j, β j, z = F j α j, β j or ˆβ = ˆβ p μ ˆβ F j α j, β j, z f F j α j, β j = F j α j, β j and ˆβ ˆβ where p = Pr ˆβ F j α j, β j The followng three clams show that μ as calculated Pr ˆβ F j α j, β j. above supports σ as a Nash equlbrum. CLAIM 3.5. μ s a vald belef system for Ɣ F. PROOF OF CLAIM 3.5. Let h be player s nformaton set after some hstory F j α j, β j, z. Clearly, μ ˆβ F j α j, β j, z 0 for all ˆβ h. We need to show μ ˆβ F j α j, β j, z = 1. ˆβ h Case 1. F j α j, β j F j α j, β j. From the constructon of F, F j α j, β j s ordered game somorphc to some F j α j β j wth F j α j β j F j α j, β j. Let h be player s nformaton set correspondng to the hstory F j α j, β j, z. Journal of the ACM, Vol. 54, No. 5, Artcle 25, Publcaton date: October 2007.

13 Lossless Abstracton of Imperfect Informaton Games 25:13 FIG. 3. Illustraton of Case 1 of Clam 3.5. By the defnton of the ordered game somorphsm, there exsts a perfect matchng between the nodes n the nformaton set h and h, where each matched par of nodes corresponds to a par of ordered game somorphc nformaton structures. Snce we have that F j α j, β j = F j α j, β j, each edge n the matchng corresponds to a node n the nformaton set correspondng to the hstory F j α j, β, z nɣ F ; denote ths nformaton set by h. See Fgure 3. Thus, there s a bjecton between h and h defned by the perfect matchng. Usng ths matchng: ˆβ h μ ˆβ F j α j, β j, z = ˆβ h = ˆβ h = 1. μ F j ˆβ F j α j, β j, z μ ˆβ F j α j, β j, z Case 2. F j α j, β j = F j α j, β j. We need to treat members of h dfferently dependng on f they map to the same set of sgnals n Ɣ F or not. Let h 1 ={ˆβ h ˆβ = F j ˆβ} and let h 2 ={ˆβ h ˆβ F j ˆβ}. Clearly, h 1, h 2 s a partton of h. Let h be player s nformaton set correspondng to the hstory F j α j, β j, z n Ɣ F. We can create a partton of h by lettng h 3 ={F j ˆβ ˆβ h 1 } and h 4 ={F j ˆβ ˆβ h 2 }. Clearly, h 3, h 4 parttons h. See Fgure 4. The rest of the proof for ths case proceeds n three steps. Step 1. In ths step, we show the followng relatonshp between h 1 and h 3 : μ ˆβ F j α j, β j, z = μ F j ˆβ F j α j, β j, z ˆβ h 1 ˆβ h 1 1 = ˆβ μ ˆβ F j α j, β j, z h 3 Step 2. In ths step we want to show a smlar relatonshp between h 2 and h 4.In dong so, we use the followng fact: ˆβ ˆβ F j ˆβ = ˆβ. Wth ths n mnd, Journal of the ACM, Vol. 54, No. 5, Artcle 25, Publcaton date: October 2007.

14 25:14 A. GILPIN AND T. SANDHOLM FIG. 4. Illustraton of Case 2 of Clam 3.5. we can wrte: μ ˆβ F j α j, β j, z = ˆβ h 2 ˆβ h 2 Pr ˆβ F j α j, β j Pr F j ˆβ F j α j, β j, z μ F j ˆβ F j α j, β j = ˆβ h 4 ˆβ h 2 ˆβ ˆβ Pr ˆβ F j α j, β j Pr F j ˆβ F j α j, β j μ F j ˆβ F j α j, β j, z = ˆβ Pr ˆβ F j α j, β j h ˆβ h 4 2 Pr ˆβ F j α j, β j 2 ˆβ ˆβ, z μ ˆβ F j α j, β j = ˆβ h 4 μ ˆβ F j α j, β j ˆβ h 2 ˆβ ˆβ Pr ˆβ F j α j, β j Pr ˆβ F j α j, β j = ˆβ h 4 μ ˆβ F j α j, β j, z, z. Step 3. Usng 2 and 3: ˆβ h μ ˆβ F j α j, β j, z = μ ˆβ F j α j, β j, z ˆβ h 1 + ˆβ h 2 μ ˆβ F j α j, β j, z = ˆβ μ ˆβ F j α j, β j, z h 3 Journal of the ACM, Vol. 54, No. 5, Artcle 25, Publcaton date: October 2007.

15 Lossless Abstracton of Imperfect Informaton Games 25:15 + ˆβ μ ˆβ F j α j, β j, z h 4 = ˆβ μ ˆβ F j α j, β j, z h = 1 In both cases we have shown μ ˆβ F j α j, β j, z = 1. ˆβ h CLAIM 3.6. For all nformaton sets h wth Prh σ > 0, μx = Prx σ Prh σ for all x h. PROOF OF CLAIM 3.6. Let h be player s nformaton set after some hstory F j α j, β j, z, and fx some ˆβ h. Let ˆβ = F j ˆβ. We need to show that μ ˆβ F j α j, β j, z = Pr ˆβ σ Prh σ. Let h be player s nformaton set after hstory F j α j, β j, z. Case 1. F j α j, β j F j α j, β j μ ˆβ F j α j, β j., z = μ ˆβ F j α j, β j = Pr ˆβ σ = Prh σ Pr ˆβ F j ˆβ F j Pr ˆβ F j ˆβ F j Pr Pr α j, β j α j, β j, z Pr ˆβ σ Prh σ α j, β j = Pr ˆβ σ Prh σ. Case 2. F j α j, β j = F j α j, β j and ˆβ ˆβ. μ ˆβ F j α j, β j Pr β F j α j, β j, z = Pr β F j α j, β j = Pr β F j α j, β j Pr β F j α j, β j = Pr β F j α j, β j Pr β F j α j, β j = Pr ˆβ σ Prh σ. α j, β j μ ˆβ F j α j, β j, z Pr ˆβ σ Prh σ Pr β F j α j, β j Pr ˆβ σ Pr β F j α j, β j Prh σ Journal of the ACM, Vol. 54, No. 5, Artcle 25, Publcaton date: October 2007.

16 25:16 A. GILPIN AND T. SANDHOLM Case 3. F j α j, β j = F j α j, β j and ˆβ = ˆβ. μ ˆβ F j α j, β j, z = μ ˆβ F j α j, β j, z = Pr ˆβ σ Prh σ = Pr ˆβ σ Prh σ. Thus, we have μx = Prx σ for all nformaton sets h wth Prh σ > 0. Prh σ CLAIM 3.7. For all nformaton sets h wth Prh σ > 0, σ s sequentally ratonal at h gven μ. PROOF OF CLAIM 3.7. Suppose, by way of contradcton, that σ s not sequentally ratonal gven μ. Then, there exsts a strategy τ such that, for some F j α j, β j, z, u j τ,σ F j α j, β j, z,μ > u j σ,σ F j α j, β j, z,μ. 3 We wll construct a strategy τ for player n Ɣ F such that u j τ,σ F j α j, β j, z,μ > u j σ,σ F j α j, β j, z,μ, thus contradctng the fact that σ s a Nash equlbrum. The proof proceeds n four steps. Step 1. We frst construct τ from τ.foragvenf j α j, β j, let ϒ = { F j α j, β j F j α j, β j F j α j, β j } 4 and let τ j,v F j α j, β j, z = ϑ ϒ Pr ϑ F j α j, β j j τ,v ϑ, z. In other words, the strategy τ s the same as τ except n stuatons where only the fltered sgnal hstory s dfferent, n whch case τ s a weghted average over the strateges at the correspondng nformaton sets n Ɣ F. Step 2. We need to show that u j τ,σ F j α j, β j, z,μ = u j τ,σ F j α j, β j, z,μ for all hstores F j α j, β j, z. Fx F j α j, β j, z, and assume, wthout loss of generalty, the equalty holds for all nformaton sets comng after ths one n Ɣ. Journal of the ACM, Vol. 54, No. 5, Artcle 25, Publcaton date: October 2007.

17 Lossless Abstracton of Imperfect Informaton Games 25:17 Case 1. F j α j, β j F j α j, β j. Let z j denote the current node of G j and let ϒ as n 4. u j τ,σ F j α j, β j, z,μ = ˆβ h μ ˆβ u j τ,σ F j α j, β j, z, ˆβ = ˆβ h = ˆβ h = ˆβ h μ ˆβ μ F j ˆβ u j τ,σ F j α j, β j j, z, F ˆβ μ ˆβu j τ,σ F j α j, β j j, z, F ˆβ v N j z j τ j,v z, F j α j, β j u j τ,σ F j α j, β j j, z, v, F ˆβ = ˆβ h μ ˆβ Pr ϑ F j α j, β j j τ,v z,ϑ v N j z j ϑ ϒ [ j u τ,σ F j α j, β j j, z, v, F ˆβ ] = ˆβ h μ ˆβ Pr ϑ F j α j, β j j τ,v z,ϑ v N j z j ϑ ϒ [ j u τ,σ F j α j, β j, z, v, ˆβ ] = ˆβ h μ ˆβ u j τ,σ F j α j, β j, z, v, ˆβ = ˆβ h μ ˆβ = ˆβ h = u j μ ˆβu j v N j z j [ ϑ ϒ v N j z j Pr ] ϑ F j α j, β j j τ,v z,ϑ τ j,v z, F j α j, β j τ,σ F j α j, β j τ,σ F j α j, β j, z,μ. u j, z, ˆβ τ,σ F j α j, β j, z, v, ˆβ Case 2.F j α j, β j = F j α j, β j. Let h 1, h 2, h 3, and h 4 as n the proof of Case 2 of Clam 3.5. We can show μ ˆβ u j τ,σ F j α j, β j, z, ˆβ ˆβ h 3 = ˆβ h 1 μ ˆβu j τ,σ F j α j, β j, z, ˆβ 5 Journal of the ACM, Vol. 54, No. 5, Artcle 25, Publcaton date: October 2007.

18 25:18 A. GILPIN AND T. SANDHOLM usng a procedure smlar to that n Case 1. We can show the followng relatonshp between h 2 and h 4 : μ ˆβ u j τ,σ F j α j, β j, z, ˆβ ˆβ h 4 = ˆβ h 4 ˆβ h 2 ˆβ ˆβ = ˆβ h 4 Pr ˆβ F j α j, β j Pr ˆβ F j α j, β j μ ˆβu ˆβ h 2 ˆβ ˆβ = ˆβ μ ˆβ h ˆβ h 4 2 ˆβ ˆβ u j τ = ˆβ h 4 μ ˆβ u j j τ,σ F j α j, β j, z, ˆβ v N j z j τ j,v z, F j α j, β j,σ F j α j, β j, z, v, ˆβ τ j,v μ ˆβ ˆβ h 2 v N j z j ˆβ ˆβ u j τ,σ F j α j, β j, z, v, ˆβ = ˆβ j μ ˆβu τ,σ F j α j, β j, z, ˆβ h ˆβ h 4 2 ˆβ ˆβ = ˆβ h μ ˆβu j τ,σ F j α j, β j, z, ˆβ. 2 τ,σ F j α j, β j, z, ˆβ z, F j α j, β j 6 Usng 6 and 7: u j τ,σ F j α j, β j, z,μ = β μ β u j τ,σ F j α j, β j, z, β h = ˆβ μ ˆβ u j τ,σ F j α j, β j, z, ˆβ h 3 + ˆβ μ ˆβ u j τ,σ F j α j, β j, z, ˆβ h 4 = ˆβ h 1 μ ˆβu j + ˆβ h 2 μ ˆβu j = ˆβ h = u j μ ˆβu j τ,σ F j α j, β j, z, ˆβ τ,σ F j α j, β j, z, ˆβ τ,σ F j α j, β j, z, ˆβ τ,σ F j α j, β j, z,μ. Journal of the ACM, Vol. 54, No. 5, Artcle 25, Publcaton date: October 2007.

19 Lossless Abstracton of Imperfect Informaton Games 25:19 FIG. 5. Example llustratng dffculty n developng a theory of equlbrum-preservng abstractons for general extensve form games. In both cases, we have shown: u j τ,σ F j α j, β j, z,μ = u j τ,σ F j α j, β j, z,μ. 7 Step 3. We can show that u j σ,σ F j α j, β j j, z,μ = u σ,σ F j α j, β j, z,μ. 8 usng a procedure smlar to the prevous step. Step 4. Combnng 3, 7, and 8, we have:, z,μ = u j τ,σ F j α j, β j, z,μ τ,σ F j α j, β j > u j σ,σ F j α j, β j u j, z,μ = u j σ,σ F j α j, β j, z,μ. Thus, σ s not a Nash equlbrum. Therefore, by contradcton, σ s sequentally ratonal at all nformaton sets h wth Pr h σ > 0. We can now complete the proof of Theorem 3.4. By Clams 3.5 and 3.6, we know that condton C2 holds. By Clam 3.7, we know that condton C1 holds. Thus, σ s a Nash equlbrum NONTRIVIALITY OF GENERALIZING BEYOND THIS MODEL. Our model does not capture general sequental games of mperfect nformaton because t s restrcted n two ways as dscussed above: 1 there s a specal structure connectng the player actons and the chance actons for one, the players are assumed to observe each others actons, but nature s actons mght not be publcly observable, and 2 there s a common orderng of sgnals. In ths subsecton we show that removng ether of these condtons can make our technque nvald. Frst, we demonstrate a falure when removng the frst assumpton. Consder the game n Fgure 5. 7 Nodes a and b are n the same nformaton set, have the same parent chance node, have somorphc subtrees wth the same payoffs, and nodes c and d also have smlar structural propertes. By mergng the subtrees begnnng at a and b, we get the game on the rght n Fgure 5. In ths game, player 1 s only Nash equlbrum strategy s to play left. But n the orgnal game, player 1 knows that node c wll never be reached, and so should play rght n that nformaton set. Removng the second assumpton that the utlty functons are based on a common orderng of sgnals can also cause falure. Consder a smple three-card game wth a deck contanng two Jacks J1 and J2 and a Kng K, where player 1 s 7 We thank Albert Xn Jang for provdng ths example. Journal of the ACM, Vol. 54, No. 5, Artcle 25, Publcaton date: October 2007.

20 25:20 A. GILPIN AND T. SANDHOLM utlty functon s based on the orderng K J1 J2 but player 2 s utlty functon s based on the orderng J2 K J1. It s easy to check that n the abstracted game where Player 1 treats J1 and J2 as beng equvalent the Nash equlbrum does not correspond to a Nash equlbrum n the orgnal game GameShrnk: An Effcent Algorthm for Computng Ordered Game Isomorphc Abstracton Transformatons In ths secton we present an algorthm, GameShrnk, for conductng the abstractons. The algorthm only needs to analyze the sgnal tree dscussed above, rather than the entre game tree. We frst present a subroutne that GameShrnk uses. It s a dynamc program for computng the ordered game somorphc relaton. 9 Agan, t operates on the sgnal tree. ALGORITHM 1. OrderedGameIsomorphc? Ɣ, ϑ, ϑ 1 If ϑ and ϑ are both leaves of the sgnal tree: a If u r ϑ z = u r ϑ z for all z r 1 ω j cont ωover r, then return true. j=1 b Otherwse, return false. 2 Create a bpartte graph G ϑ,ϑ = V 1, V 2, E wth V 1 = Nϑ and V 2 = Nϑ. 3 For each v 1 V 1 and v 2 V 2 : If OrderedGameIsomorphc? Ɣ, v 1, v 2 Create edge v 1, v 2 4 Return true f G ϑ,ϑ has a perfect matchng; otherwse, return false. By evaluatng ths dynamc program from bottom to top, Algorthm 1 determnes, n tme polynomal n the sze of the sgnal tree, whether or not any par of equal depth nodes x and y are ordered game somorphc. The test n step 1a can be computed n O1 tme by consultng the relaton from the specfcaton of the game. Each call to OrderedGameIsomorphc? performs at most one perfect matchng computaton on a bpartte graph wth O nodes and O 2 edges recall that s the set of sgnals. Usng the Ford-Fulkerson algorthm [Ford, Jr. and Fulkerson 1962] for fndng a maxmal matchng, ths takes O 3 tme. Let S be the maxmum number of sgnals possbly revealed n the game e.g., n Rhode Island Hold em, S = 4 because each of the two players has one card n the hand plus there are two cards on the table. The number of nodes, n, n the sgnal tree s O S. The dynamc program vsts each node n the sgnal tree, wth each vst requrng O 2 calls to the OrderedGameIsomorphc? routne. So, t takes O S 3 2 = O S+5 tme to compute the entre ordered game somorphc relaton. Whle ths s exponental n the number of revealed sgnals, we now show that t s polynomal n the sze of the sgnal tree and thus polynomal n the sze of the game tree because the sgnal tree s smaller than the game tree. The number of 8 We thank an anonymous person for provdng ths example. 9 Actually, ths s computng a slghtly relaxed noton snce t allows nodes wth dfferent parents to be consdered ordered game somorphc. However, the GameShrnk algorthm only calls t wth sblng nodes as the arguments. Journal of the ACM, Vol. 54, No. 5, Artcle 25, Publcaton date: October 2007.

21 Lossless Abstracton of Imperfect Informaton Games 25:21 nodes n the sgnal tree s n = 1 + S 1 j + 1 j 1 Each term n the summaton corresponds to the number of nodes at a specfc depth of the tree. The number of leaves s S j + 1 = S! S j=1 whch s a lower bound on the number of nodes. 10 For large we can use the relaton n k n k k! to get S S! S S! S! = S and thus the number of leaves n the sgnal tree s S. Therefore, O S+5 = On 5, whch proves that we can ndeed compute the ordered game somorphc relaton n tme polynomal n the number of nodes, n, of the sgnal tree. The algorthm often runs n sublnear tme and space n the sze of the game tree because the sgnal tree s sgnfcantly smaller than the game tree n most nontrval games. Note that the nput to the algorthm s not an explct game tree, but a specfcaton of the rules, so the algorthm does not need to read n the game tree. In general, f an ordered game has r rounds, and each round s stage game has at least b nontermnal leaves, then the sze of the sgnal tree s at most 1 of the sze of b the game tree. For example, n Rhode Island Hold em, the game tree r has 3.1 bllon nodes whle the sgnal tree only has 6,632,705. Gven the OrderedGameIsomorphc? routne for determnng ordered game somorphsms n an ordered game, we are ready to present the man algorthm, GameShrnk. ALGORITHM 2. GameShrnk Ɣ 1 Intalze F to be the dentty flter for Ɣ. 2 For j from 1 to r: For each par of sblng nodes ϑ, ϑ at ether level j 1 k=1 κ k + nγ k or j k=1 κk + j 1 k=1 nγ k n the fltered accordng to F sgnal tree: If OrderedGameI somor phc?ɣ, ϑ, ϑ, then F j ϑ F j ϑ F j ϑ F j ϑ. 3 Output F. Gven as nput an ordered game Ɣ = I, G, L,,κ,γ,p,,ω,u, GameShrnk apples the shrnkng deas presented above as aggressvely as possble. Once t fnshes, there are no contractble nodes snce t compares every par of nodes at each level of the sgnal tree, and t outputs the correspondng nformaton flter F. The correctness of GameShrnk follows by a repeated applcaton of Theorem 3.4. Thus, we have the followng result: 10 Usng the nequalty n k n k k, we get the lower bound S S! S S S! = S S!. S S Journal of the ACM, Vol. 54, No. 5, Artcle 25, Publcaton date: October 2007.

22 25:22 A. GILPIN AND T. SANDHOLM THEOREM 4.1. GameShrnk fnds all ordered game somorphsms and apples the assocated ordered game somorphc abstracton transformatons. Furthermore, for any Nash equlbrum, σ, of the abstracted game, the strategy profle constructed for the orgnal game from σ s a Nash equlbrum. The domnatng factor n the run tme of GameShrnk s n the rth teraton of the man for-loop. There are at most S S! nodes at ths level, where we agan take S to be the maxmum number of sgnals possbly revealed n the game. Thus, the nner for-loop executes O 2 S S! tmes. As dscussed n the next subsecton, we use a unon-fnd data structure to represent the nformaton flter F. Each teraton of the nner for-loop possbly performs a unon operaton on the data structure; performng M operatons on a unon-fnd data structure contanng N elements takes OαM, N amortzed tme per operaton, where αm, N s the nverse Ackermann s functon [Ackermann 1928; Tarjan 1975] whch grows extremely slowly. Thus, the total tme for GameShrnk s O 2 S S! α 2, S S! S. By the nequalty n k n k k!, ths s O S 2 α S 2, S. Agan, although ths s exponental n S, tsõn 2, where n s the number of nodes n the sgnal tree. Furthermore, GameShrnk tends to actually run n sublnear tme and space n the sze of the game tree because the sgnal tree s sgnfcantly smaller than the game tree n most nontrval games, as dscussed above EFFICIENCY ENHANCEMENTS. We desgned several speed enhancement technques for GameShrnk, and all of them are ncorporated nto our mplementaton. One technque s the use of the unon-fnd data structure [Cormen et al. 2001, Chap. 21] for storng the nformaton flter F. Ths data structure uses tme almost lnear n the number of operatons [Tarjan 1975]. Intally each node n the sgnallng tree s ts own set ths corresponds to the dentty nformaton flter; when two nodes are contracted they are joned nto a new set. Upon termnaton, the fltered sgnals for the abstracted game correspond exactly to the dsjont sets n the data structure. Ths s an effcent method of recordng contractons wthn the game tree, and the memory requrements are only lnear n the sze of the sgnal tree. Determnng whether two nodes are ordered game somorphc requres us to determne f a bpartte graph has a perfect matchng. We can elmnate some of these computatons by usng easy-to-check necessary condtons for the ordered game somorphc relaton to hold. One such condton s to check that the nodes have the same number of chances as beng ranked accordng to hgher than, lower than, and the same as the opponents. We can precompute these frequences for every game tree node. Ths substantally speeds up GameShrnk, and we can leverage ths database across multple runs of the algorthm e.g., when tryng dfferent abstracton levels; see next secton. The ndces for ths database depend on the prvate and publc sgnals, but not the order n whch they were revealed, and thus two nodes may have the same correspondng database entry. Ths makes the database sgnfcantly more compact. For example n Texas Hold em, the database s reduced by a factor / 50 5 = 20. We store the hstograms n a 2- dmensonal database. The frst dmenson s ndexed by the prvate sgnals, the second by the publc sgnals. The problem of computng the ndex n ether one of the dmensons s exactly the problem of computng a bjecton between all Journal of the ACM, Vol. 54, No. 5, Artcle 25, Publcaton date: October 2007.