ATHABASCA UNIVERSITY CAN TEST DRIVEN DEVELOPMENT IMPROVE POKER ROBOT PERFORMANCE? EDWARD SAN PEDRO. An essay submitted in partial fulfillment

Transcription

1 ATHABASCA UNIVERSITY CAN TEST DRIVEN DEVELOPMENT IMPROVE POKER ROBOT PERFORMANCE? BY EDWARD SAN PEDRO An essay submitted in partial fulfillment Of the requirements for the degree of MASTER OF SCIENCE in INFORMATION SYSTEMS Athabasca, Alberta March, 2008 Edward San Pedro, 2008

2 ABSTRACT Is it possible to create a poker playing robot that can beat a professional human player? Researchers are attempting to answer this question with a yes. The game of poker is extremely popular and it has caught the attention of the computer science community. We believe that some practical approaches to software development can potentially improve the performance of poker robots. Poker robot development can benefit from the application of Test Driven Development and performance testing. This essay investigates the current research that is available for poker robot development and Test Driven Development. To help support the potential benefits of combining testing with poker robot development, a simple poker robot is modified with Test Driven Development and performance testing techniques and tools. This programming exercise helps expand the research results by applying some of the theory to a hands-on situation. The results of this exercise provides evidence to support the use of Test Driven Development in improving poker robot performance. Poker robot developers may find Test Driven Development to be a very useful approach to designing their software. However, Test Driven Development is only one of many potential tools that can be used. ii

3 ACKNOWLEDGMENTS I would like to thank Mai for her support. She also provided me with the inspiration to create a unique poker robot name. I would also like to thank my essay supervisor Dr. Dunwei Wen for his patience and guidance. A final thank you to the School of Computing and Information Systems for their support and influence. iii

4 Table of Contents Chapter I... 1 Introduction... 1 Chapter II... 3 Poker and Artificial Intelligence... 3 Artificial Intelligence and Poker Overview... 3 Test-driven Development... 6 Test-Driven Development Research Overview... 6 Applying TDD to Poker robot development... 8 Poker Robot Design Using Test-Driven Development and Performance Testing Chapter III Poker Robots and Test-Driven Development Current State of Poker Robot Development Hold'em Poker Current State of Test-Driven Development Poker robot development and TDD Chapter IV Research Study Results Applying TDD and Performance Testing to Poker Robot Development Poker Robot Development with TDD iv

5 TDD and Unit Testing Performance Testing Poker Robot and Test-Driven Development Discussion Chapter V Research Implications Research Results Limitations Rationale for the research Chapter VI Conclusions References Appendix A Program documentation Running Project tests Appendix B - Poker Hands v

6 vi

7 CHAPTER I INTRODUCTION Poker is an extremely popular game. Many people are drawn to this card game. Many poker tournaments are televised and some successful professional players are famous. Poker has also captured the interest of the computer science and artificial intelligence community [1]. Poker playing programs have been written, which are called poker robots. The game of poker presents challenging problems in the quest to create a robot that can player their hands strongly. Research has been conducted with regards to poker robots playing at a competitive level. Is it possible to improve on existing poker robot designs and algorithms? Test Driven Development (TDD) is a software development approach that focuses on testing [2, 3]. This approach to software development is being used in university course work. TDD is also being used by software professionals. There are additional development tools that support TDD and performance testing. Since poker involves decision making that should be done within a reasonable amount of time, it would be interesting to investigate how useful TDD and performance testing would be in poker robot development. This essay analyzes this subject and provides useful results that support the use of TDD and performance testing in poker robot development. 1

8 Research in the fields of poker, artificial intelligence, TDD, and performance testing was conducted to identify potential connections between the subject matter. In order to illustrate the potential use of TDD and performance testing in poker robot development, an existing poker robot was modified. The results of this research show that TDD and performance testing are very useful practices to use when developing poker robots. The format of the essay is as follows. Chapter II provides an introduction to poker robot development and artificial intelligence, TDD and performance testing. In chapter III, this essay digs deeper to look at the existing research in poker robot development and artificial intelligence, TDD, and performance testing. This chapter also provides more insight into the rules of poker and characteristics of strong players. This chapter discusses the potential benefits and challenges of applying TDD and performance testing to poker robot development, which leads into Chapter IV, which provides a discussion on research results that include an exercise in applying TDD and performance testing to a simple poker robot design. Chapter V looks at the implications of the research results from chapter IV. 2

9 CHAPTER II POKER AND ARTIFICIAL INTELLIGENCE Artificial Intelligence and Poker Overview Poker popularity is currently at a high level. Poker shows are frequently found on television. Poker playing is a global phenomenon. Many world class players can be found all over the world. With high poker interest and the growth of the Internet, on-line poker playing continues to grow [1]. Many on-line poker players play at varying stakes from penny antes to high limit games. On-line poker rooms reach people that do not normally have access to casinos. Poker has also caught the interest of the computer science community. The nature of poker makes it an interesting subject for artificial intelligence (AI) research [4]. A poker game provides an interesting environment for AI since each player conceals the hand that he or she holds. The University of Alberta (U of A) believes that their poker research shows that they are close to developing a computer program that can compete at a world class level [5]. Interest in applying AI to the poker game attracts so much interest that a poker competition called the World Series of Poker Robots was held in Las Vegas 3 years ago [1]. 3

10 The University of Alberta Computer Poker Research (Poker Research) group states that its poker AI is "the strongest in the world [4]". This research group's web-site provides many references to its publications and other useful links related to poker. This group focused its research on poker because the game "is an interesting test-bed for artificial intelligence research [6]." Poker provides challenging artificial intelligence research because it involves imperfect information, "where multiple competing agents must deal with risk management, agent modeling, unreliable information and deception, much like decision-making applications in the real world [6]." Chess and checkers are examples of perfect information games because the game board holds all the information required, which is visible to both participants. Computer AI research in perfect information games, such as chess, results in brute-force searches. Researchers discovered that a strong chess playing computer program must be able to calculate moves quickly. One U of A paper [7] reports that IBM's Deep Blue computer calculates 250 million chess positions per second. Many card games provide examples of imperfect information games, such as bridge and poker. Poker is an imperfect information game because each player in the game should not know the cards that other players hold. A brute-force search is unsuccessful in imperfect information games because "it is often impractical to search the game trees that result from all possible instantiations of the missing information [7]." The Poker Research group decided to use the Texas Hold'em (Hold'em) poker variant for AI research. 4

11 Progress in poker AI research may provide benefits to other situations with imperfect information. There are many situations where complete information is not available, such as forecasting product demand, or managing company purchasing by guessing the strategies used by business rivals [8]. This research may also provide benefits for the general public. One example is the use of AI to provide investing guidance. This type of software may help out people that normally would not have the opportunity to participate in investing. Other interesting imperfect information situations include political issues, conflict mediation, legal issues, and project management. As computers provide increasing processing power, poker AI algorithms will be able to conduct more calculations for poker playing purposes. This ability to make additional computations may alter the current poker AI algorithms. Researchers may need to look at adjusting the algorithms as more processing power is made available. In contrast, researchers may also try to investigate shortcuts in their poker AI algorithms, which the U of A Poker Research group has already started [5]. Research into efficient algorithms may discover areas that may not be necessary, or may provide no practical benefits. These areas can then be removed or altered as necessary. Poker AI research should also be examined for other real world applications. If the poker AI algorithms can be used to benefit other fields of study, then researchers may 5

12 have another source of effective techniques for implementing these algorithms in other imperfect information games. If artificial intelligence research involving software can be improved through the use of performance testing, then TDD should be reviewed by researchers. TEST-DRIVEN DEVELOPMENT Test-Driven Development Research Overview Test-Driven Development (TDD) refers to the software development approach where the programmers write tests before writing production code. With TDD, the same programmer writes the test and production code. The tests serve the purpose of driving the design and implementation of the development tasks. TDD is performed in small iterations. First, the programmer runs a test to show that it fails. Second, the programmer writes just enough production code to make the test pass. Third, the test is executed again to show success. Finally, the programmer examines the code to identify any possible improvements to the production code, such as removing code duplication. This process is repeated for every cycle. Each time another test is added, all existing tests are executed. TDD also serves regression testing purposes, which is a valuable part of maintaining software quality and reducing risks with change management [9, 10]. 6

13 Programmers can use software tools and development environments to help with the TDD process. These tools make it easier for programmers to create and execute tests. Tools that promote TDD and testing practices can help programmers focus on creating high quality software solutions. TDD can help to reduce bugs. Debugging can take up a lot of time to track down and fix. A suite of tests that can be executed easily can reduce the risk of too much debugging [11]. Extreme programming (XP) is a software development approach that aims to follow strict programming practices. A large part of following XP requires the software team to develop unit tests and continuously integrate newly developed code into the overall software project. For example, some traditional software approaches may focus on breaking down the overall project, developing the smaller parts and integrating them as a final step. With XP, the overall project is also broken down, but integration takes place right after a piece of code is complete. In XP, integration is a continuous process, which helps to ensure the entire project works fully. An important part of XP that contributes to the success of continuous integration is unit testing. In XP, a software team designs unit tests to test individual methods and classes. For example, if a software program is written in Java, the unit tests are also written in Java [2]. Unit tests should be written for any complex code and any nontrivial implementation of a method. Ideally, a unit test should be written for anything that might 7

14 break. In XP, unit tests must always pass. This requirement helps to ensure that continuous integration does not break any functionality that currently works. Next, the discussion moves on to exploration of extreme programming uses. APPLYING TDD TO POKER ROBOT DEVELOPMENT TDD provides many useful techniques that can help with the progress of poker robot research. This essay will now discuss the application of TDD to poker robot development. The TDD approach will focuses on incrementally adding functionality to software. In the case of poker robots, TDD will force developers to create test cases that demonstrate how the poker robot will function. These test cases help to reinforce initial design decisions, or challenge assumptions. As the poker robot code base grows, the number of test cases increases. These test cases provide important feedback whenever changes are made to the source code. If a test case suddenly starts to fail when a change is made, the developer can review the change to figure out how it caused the test to fail. This approach to poker robot development can help to increase the quality of the source code [12]. TDD can also act as an important documentation tool. Test cases verify the functionality of the source code. Developers read the test case code and they gain an understanding of the functionality and the design under test. Although test cases may not 8

15 completely meet the documentation needs of a poker robot product, these tests evolve with the design of the source code. In the event that existing documentation may not be up to date, the test cases can be used to help developers gain an understanding of a poker robot developed with TDD [13]. TDD may also provide benefits from the perspective of poker robot research [14]. The regression testing and incremental design implementation may make it easier to transfer knowledge to other researchers. Different researchers may be involved in the development of a poker robot. In the university setting, development may occur at different times. The test cases and the source code provide insight into the design of the poker robot. The test case code show the practical application of the theory behind poker robot development. TDD may help researchers improve the application of theory to poker robot programming. If the TDD approach includes performance testing, developers have ways to quantify acceptable poker robot performance. A poker robot that can play more hands in a given time frame will generate more data as well. Researchers can use this data to determine the playing strength of the poker robot. This quick turnaround in feedback can lead to fast implementation of changes. Speeding up the poker robot development can lead to greater progress in this area of research. The benefits of applying TDD and performance testing to the area of poker robots may also be useful in other areas of computer science and artificial intelligence [15, 16]. 9

16 The research for this essay consists of two stages. First, supporting literature was found for the two main research topics of poker robot development and TDD. The TDD research includes performance based testing. Second, a simple poker robot was designed to evaluate the testing tools available for poker robot software development. The poker robot development testing results provide useful information on how performance testing may benefit future research in artificial intelligence. The program described in this essay helps to illustrate the TDD and performance test process as it may apply to poker robot research. Poker Robot Design Using Test-Driven Development and Performance Testing An existing poker robot program exists with the Poker Academy software product [17]. The program is written in the Java programming language. The JUnit testing framework is used to run the tests created for this program [18]. JUnitPerf performance tests are available for this program [19]. The poker robot program was modified so that JUnit and JUnitPerf tests could be created. These tests serve the purpose of verifying that the program behaves as expected. The Eclipse development environment software was used to write the software associated with this poker robot [20]. Eclipse provides testing tools that allow programmers to execute JUnit and JUnitPerf tests. 10

17 CHAPTER III POKER ROBOTS AND TEST-DRIVEN DEVELOPMENT Current State of Poker Robot Development Poker robot development poses a number of different challenges. First, the development team must have a good understanding of the rules of the game. Many poker robot researchers have a strong interest in the game. Some researchers grew up playing the game with their families. Other people have experience playing poker in casinos or card rooms. The Internet also provides many people with a convenient option if they would like to play cards. Second, poker robot developers must understand the skills that are required to be a good player. A good poker player must play with a positive expectation. Successful poker playing can be considered both an art and a science. Players must be able to assess the strengths and weaknesses of their opponents. These players must also understand the mathematical aspect of the game. Third, meaningful results from poker robot research can only be produced if a poker robot is programmed to make decisions quickly. Poker robot decisions on poker hands should take no more than a few seconds at most [21]. What is required for a poker bot to become a strong poker player? One, a strong poker player knows how to bet appropriately. Two, a strong player can determine the 11

18 playing styles of their opponents, which is opponent modeling, and use this information to exploit weaknesses. Three, a strong poker player must be disciplined to play winning poker. If a player allows emotion to affect their play, then they may play poorly. Successful poker playing involves a number of different strategies, such as deception, opponent modeling, and dealing with uncertainty [22]. The U of A Poker Research group uses mathematics and a random number generator to determine betting strategies for its poker bot. The Poker Research group uses game theory to create a near optimal betting strategy for its poker bot [5]. The Poker Research group implements opponent modeling by using statistical methods. Data on how opponents play are saved and used in subsequent poker playing. The Poker Research group admits that opponent modeling is "a limiting factor to the overall play [23]." Poker bots can play poker without emotion becoming a factor. This quality of a strong poker player was not a concern to the Poker Research group. The poker AI research conducted by the U of A provides the necessary information to make a prediction on what the future holds for poker bots. The U of A poker AI research suggests that poker bots without opponent modelling can already defeat strong human players and be competitive against world-class opposition [5]. One version of a poker bot developed by the U of A played against a world-class human player. The human player acknowledged that the poker bot was a challenging opponent [5]. This 12

19 statement provides an opinion that poker bots may be able to playing winning poker against world-class opposition in the future. Poker programs are showing improvements in playing the game, which includes ways to handle different forms of uncertainty [21]. Other poker players have different opinions. Professional poker player and Card Player magazine columnist Barry Tanenbaum states: "In a mathematical sense a computer can assess its chances. But it can't see someone twitch." Poker expert David Sklansky also believes that a program can theoretically play the game at a reasonably high level, but he believes that a computer will never be able to beat him. Book writer Alan Schoonmaker counters by stating that computers can be programmed to play optimally at all times. Schoonmaker concludes that computers will eventually beat the poker equivalent of a chess grandmaster [8]. Strong poker players believe that trends can be recognized from observing an opponent's betting pattern. These trends are easier to observe where there are only 2 or 3 players in a game. For larger games, more general trends are observed, since it can be difficult to keep track of 8 or 9 opponents. This information leads to the belief that a poker bot can be developed to play strong poker at a high level. However, due to the luck aspect of the game, it may be difficult to conclude that a poker bot will be able to play at a world-class level without extensive data of play against world-class opposition. 13

20 The U of A based poker AI research that evolved into the Poker Academy product may only need a little adjustment to their poker playing algorithms to play poker at a world-class level. The only people that really know the strength of play by the poker AI are the programmers involved with Poker Academy development. However, the general public may find out about the progress of the Poker Academy AI in the near future. Poker Academy boasts of endorsements from professional poker players [17]. Having purchased the Poker Academy product, the author of this report believes that the AI in the software is already very strong and that this version of the AI is probably not the optimized version. The optimum poker bot AI may already be capable of soundly defeating strong opposition. Since brute-force methods are not useful in poker, opponent modeling algorithms are required. If these algorithms have already reached their limitations, then it may be difficult to create a poker bot that plays at a world-class level. This discovery may be a possibility, but seems highly unlikely due to the results of the research from the U of A Poker Research group. To provide strong evidence to support the playing strength of poker bots, these robots need to be matched up against many different world-class players. The winning percentage of the poker bots will show whether the poker AI algorithms can compete at a world-class level. The poker bots must play many hands against world-class opposition 14

21 for the results to represent valid trends. One of U of A's poker bots played 7000 hands against a world-class player [5]. Although this sample did not provide statistically conclusive results, the world class player showed winning results against the poker bot. However, the poker bot provided the world class player with some challenging play. The U of A research team believes that a strong player should be able to consistently beat the poker robot. The U of A sample size shows that poker bots should be put into play as much as possible. Having poker bots play more hands means more meaningful data points for examination. The poker research conducted by the University of Alberta provides high level aspects to consider when designing a poker robot. First, a poker playing strategy should be implemented. This betting strategy implements the decision making requirements for a given poker game. The University of Alberta focused on the poker game of Hold 'Em. Second, an opponent modeling module should be implemented. Opponent modeling provides the poker robot with information on how their opponents play. This information can be exploited that may increase potential profitability [24]. Poker robot playing strategies can be divided into four main categories. First, a heuristic based approach can be used. In this approach, the poker playing is based on a set of rules. Second, a simulation based approach is possible. In this approach, a poker robot would run simulations for each possible action. Then the poker robot would be 15

22 programmed to select one of these actions, based on profitability. The third category involves game theory. Using game theory is an effective strategy for poker because it can be used to achieve the best case results against a worst case adversary. A game theory strategy can also be known by an opponent, but the opponent cannot gain an advantage or exploit this knowledge. Finally, the last category is a heuristic search based approach to poker playing. This approach attempts to plan a playing strategy by performing backward induction. Backward induction is used to consider the various end states for a given poker hand. Each potential action is given a value according to the corresponding end states. The given values provide the poker robot with an appropriate action to take [24]. Opponent modeling can be implemented in different ways. First, a strategy model attempts to learn an opponent's behaviour in poker playing. A strategy model maps an opponent's decision making to a probability distribution over the opponent's available play options. Second, an observation model is used where the playing decisions are based on the information available to the decision maker at each point in the poker game. An observation model is created by making attempts to measure the probability of observing hypothetical game actions [24]. Hold'em Poker Poker is a card game, usually played for money. Many different variants of poker exist, but one variant that receives a lot of attention is Hold'em poker. Hold'em poker is 16

23 played to decide the winner of the main event of the World Series of Poker. Hold'em poker is played by dealing each player two hole cards that are dealt face down. After the two hole cards are dealt, there is a betting round. Then, three community cards are dealt face up. These three cards are known as the flop. After the flop is dealt, there is another betting round. Then, another community card is dealt face up, which is known as the turn. Now, another betting round is conducted. After this betting round is complete, a fifth and final community card is dealt face up, which is known as the river. Then, a final betting round is conducted. Once the final betting round is complete, any remaining players show their cards to determine the winner. The winner holds the best five-card poker hand. The winner then takes all the money accumulated from the betting rounds, also known as the pot. Hand rankings are available in the appendix. These rankings provide an idea of the odds associated with the different poker hands [8]. Hold'em poker, like any other poker variant, involves a mixture of skill and luck. Poker theory is based in mathematics. Any hold'em poker player has the ability to calculate their chances of forming a poker hand by analyzing the cards that they hold, the community cards that are dealt face up, and the number of cards that are unknown. The unknown cards consist of the cards dealt to other players and the cards that remain in the deck. With all of this card information, a player can calculate their chances of forming a hand using probability theory. However, mathematics is only one aspect of poker. To become successful, a poker player must also be able to determine how their opponents 17

24 play the game. Winning poker players are strong opponent modellers. Opponent modelling provides great challenges to the AI research community. Poker resources on the Internet show that there is interest in developing strong poker playing AI software. A poker robot that is developed to play Hold'em will be based on the betting rounds during a single poker hand. First, a pre-flop decision model should be implemented. Pre-flop decisions are based on the the two hole cards that are dealt to a player. Second, a decision model should be implemented for the betting rounds after the flop, or post-flop. The post-flop decisions involve actions immediately after the flop, after the turn (fourth community card), and the river (fifth and final community card). These decisions complete the betting action requirements in Hold'em. A poker robot does not require opponent modeling to play poker. Opponent modeling allows a poker robot to play a stronger game, which may result in positive expectation results. In a regular casino poker game, decision making is usually made within one minute. In an on-line poker game, decisions must be made within a time limit that is usually under one minute. Poker robots that play against other poker robots have the flexibility to take any amount of time to make decisions. However, a poker robot that can make faster decisions provides researchers with more results of their play, in a shorter period of time. 18

25 Current State of Test-Driven Development Software development with TDD is researched and studied in universities and colleges. Many software projects are managed by professionals. The source code is managed in many different ways, depending on the people involved. Practical application of TDD is also supported by software tools that are available for many different programming languages. Test-driven development has been gaining popularity over the past few years. Some of the TDD techniques that are described in articles have been employed by many experienced software developers over the last twenty years [18]. Some software engineering research focuses on the overall benefits of TDD. Effective use of TDD can lead to a meaningful decrease in defects. A study at North Carolina State University shows that code developed using TDD showed 40% fewer defects when compared to a baseline product that was developed using a more traditional approach. Traditional methods of software development include requirements gathering, coding and debugging, and automated or manual testing. As mentioned earlier, the main difference between TDD and traditional software development is that TDD involves test writing before production source code is written. A software development team at IBM incorporated TDD practices into one of their projects. The project started with a UML diagram design process. From the resulting UML diagrams, the development team wrote code using TDD development. The IBM development team was relatively inexperienced, but they had strong leadership and support to help with adopting the TDD process [10]. 19

26 TDD also promotes increased test writing and coverage, which can also improve productivity. A study showed that the measured quality of a program increased with the number of programmer tests written [11]. Since TDD promotes iterative software development, programmers can also program at a rapid pace. Programmers can also become agile, by effectively handling changes to requirements. The use of automated and regression test runs promotes a sense of confidence in making changes to a software project. The test runs provide the required verification to avoid disruption in a software project [13]. Some experienced software developers believe that TDD promotes a professional approach to software development. When TDD is combined with software project management, the software development tasks can be analyzed as it relates to the tests. The source code can be analyzed to determine the number of tests and the code coverage for production code. These metrics can help the development team identify areas in the code base that require test coverage. TDD promotes a proactive approach to the software verification process [12]. A number of tools exist to aid in the adoption of TDD. These software development tools provide programmers with a test framework that makes test writing easier. For the Java programming language, the JUnit test framework can be used to write 20

27 tests [18]. These tests are structured so that they can be executed in an automated fashion. The Eclipse integrated development environment (IDE) supports the JUnit testing framework [20]. Eclipse is a powerful and popular Java development tool. A performance testing tool is also available for Java. JUnitPerf is a performance testing framework that gives Java programmers the ability to measure the performance of JUnit tests. The combination of Eclipse, JUnit, and JUnitPerf provide the necessary tools to create performance tests for a poker robot. Poker robot development and TDD Using TDD with poker robot development can help researchers focus on measuring and improving performance. First, tests can be created to verify that a poker robot makes the expected decisions when playing a specific hand. Second, once tests are created for a poker robot, these tests can be analyzed for performance. These performance tests can be used to measure the time it takes for a poker robot to make decisions. It can also measure other modules of a poker robot. These performance test results can be combined to provide a more complete picture of the time it takes for a poker robot to complete a decision in a game. 21

28 CHAPTER IV RESEARCH STUDY RESULTS Applying TDD and Performance Testing to Poker Robot Development The literature reviewed concerning poker artificial intelligence research shows that there may be benefits that extend past the poker game. Since poker is a game with imperfect information, it could be useful in other real life situations. One potential benefit is in the investing field. Investing research involves a number of different variables. The artificial intelligence in poker may provide useful models that can be used in investing (reference). The opponent modeling component in poker robots can also be used in other applications. The Poker Academy poker software [17] provides an application programming interface for creating poker robots. Poker Academy also provides the Java source code for a basic poker robot, SimpleBot.java. SimpleBot will not be assessed for its quality of poker playing. It will be used as the basis for JUnit and JUnitPerf test case writing. The main focus will be on the potential benefits of creating JUnitPerf performance tests for SimpleBot. SimpleBot was used as a starting point to create another poker robot, MaiBot. The robots were written in the Java programming language. These poker robots can be used with the Poker Academy software. Poker Academy allows the user to plug-in poker robots to the software. The poker robots then play hands with other robots that are already 22

29 available in Poker Academy. Alternatively, human players can play against poker robots in Poker Academy. MaiBot will be written using TDD. MaiBot is compatible with Poker Academy version 2 [17]. SimpleBot is programmed to play the Hold'em variant of poker. The SimpleBot source code was written by an employee of BioTools, the company that created Poker Academy. The decisions that it makes can be divided into two main sections. The first decision is related to pre-flop action and the second decision relates to post-flop action. SimpleBot is programmed to play the fixed limit Hold'Em variant of poker. This poker robot was provided by the company that sells Poker Academy. SimpleBot is available with the Java API, Meerkat, that supports development for Poker Academy. The Meerkat API provides all the necessary details to plug a poker robot into Poker Academy to play poker. SimpleBot consists of one file and class, SimpleBot.java. This class contains all of the required decision making logic for playing poker. SimpleBot contains a method, preflopaction(), that determines SimpleBot's decision before the flop. SimpleBot's method, postflopaction(), determines the decisions to make on the three betting rounds: after the flop, the turn, and the river. The main goal behind MaiBot development is to illustrate the use of TDD and performance testing with poker robot development. The Poker Academy software provides the necessary environment for poker playing robots. Along with the Meerkat 23

30 API, this software features different poker robots that can serve as opponents to human opponents, or custom robots, such as MaiBot. Games between robots can be automated, so hands can be played between robots without any human intervention. Poker Robot Development with TDD The MaiBot decision making process is exactly the same as SimpleBot's. For the purposes of this essay, nothing was changed with respect to SimpleBot's decision making. MaiBot differs from SimpleBot in a number of ways. Using the TDD approach, the SimpleBot design was broken into different classes. The SimpleBot functionality was analyzed and JUnit test cases were created for MaiBot. Following the TDD approach, MaiBot source code was written to made the JUnit tests pass. The incremental TDD cycles of writing a test, writing code, and verifying successful test runs were followed throughout the MaiBot development process. JUnit tests were created for the decision making classes within the MaiBot design. At this point, the MaiBot source code was analyzed for performance testing. This process involved the use of JUnitPerf to create tests that measured the execution time of the JUnit tests that were created earlier. At the end of this exercise, the MaiBot source code provides the same type functionality that the original SimpleBot exhibits. However, MaiBot includes a number of tests that can be executed easily. MaiBot also features a few performance tests that show that some of its methods execute under a given time threshold. 24

31 Development on MaiBot started by breaking down SimpleBot into smaller classes. Since SimpleBot already provided a fully functioning poker robot that can be used with Poker Academy, the focus was to improve the design. The poker play decision making in SimpleBot was analyzed. SimpleBot contains logic to decide on pre-flop and post-flop actions. To begin development on MaiBot, two test classes were created, PreFlopModelTest and PostFlopModelTest. These test classes contain the JUnit tests that are used to test these two MaiBot classes : PreFlopModel and PostFlopModel. These two classes contain the SimpleBot methods that handle the appropriate decision making actions. This decision was made to make the MaiBot design more intuitive and easier to follow than SimpleBot's. A third test class, HoldEmActionTest, was created to test the MaiBot class, HoldEmAction. HoldEmAction consumes the two decision making classes. The JUnit tests were written and executed using the eclipse development software. Figures 1 to 3 shows the MaiBot project with the three test classes and 38 tests included in a successful JUnit test run. 25

32 Figure 1: Eclipse IDE Unit Test Run, HoldEmActionTest.java 26

33 Figure 2: PreFlopModelTest.java. 27

34 Figure 3: PostFlopModelTest.java. The following table summarizes the tests created for MaiBot. Test Class and Tests PreFlopModelTest/HoldEmActionTest testpreflopaces testpreflopkings testpreflopqueens testpreflopjacks testprefloptens testpreflopnines testpreflopactionacekingspades testpreflopactionqueentenhearts testpreflopactionqueentenoff Description Verify raise with this hand. Verify raise with this hand. Verify raise with this hand. Verify raise with this hand. Verify raise with this hand. Verify call with this hand. Verify raise with this hand. Verify raise with this hand. Verify call with this hand. 28

35 Test Class and Tests testpreflopaction87diamonds testpreflopaction23clubs testpreflopactionace2suited testpreflopactionace8suited testprefloprandomfold testprefloprandomcall testprefloprandomcheck testpreflopaction PostFlopModelTest testpostflopactionaces testpostflopactionkings testpostflopactionqueens testpostflopactionjacks Description Verify call with this hand. Verify call with this hand. Verify raise with this hand. Verify call with this hand. Verify threshold for random fold. Verify threshold for random call. Verify threshold for random check. Verify threshold for random check. Verify raise post flop. Verify raise post flop. Verify raise post flop. Verify raise post flop. All of the tests listed above run with zero failures. The tests created in the HoldEmActionTest class serve as integration tests to verify that the PreFlopModel class was integrated successfully with the HoldEmAction class. A Poker project was created in eclipse. This list of files were included in this project: HoldEmAction.java HoldEmActionTest.java MaiBot.java PostFlopModel.java 29

36 PostFlopModelTest.java PreFlopModel.java PreFlopModelTest.java In addition to the Java source files, some references were added, meerkat-api.jar and junit.jar. The meerkat-api.jar file provides the API that is required to integrate MaiBot with the Poker Academy program. The junit.jar file provides the necessary API for the test classes. The PreFlopModelTest class tests the pre-flop decisions that are expected from the PreFlopModel class. A number of different hole card combinations are provided in separate tests. The PostFlopModelTest class tests the post-flop decisions that are expected from the PostFlopModel class. Like the pre-flop decisions, a number of different hole card combinations are provided in separate tests. TDD and Unit Testing One main goal behind MaiBot development was to write the source code according to the principles of TDD. TDD was applied to MaiBot development in a step by step manner. This is how TDD was interpreted for the purposes of MaiBot development: 1. Functionality would be provided by the SimpleBot source code. 30

37 2. Unit tests would be written first, before writing MaiBot source code. 3. MaiBot source code are written to make the tests pass. 4. The MaiBot source code was then reviewed to remove code duplication and to break down large methods into multiple smaller methods that are more manageable (refactoring). These steps were followed incrementally until all unit tests were written. The existing SimpleBot source code that is provided by Poker Academy was redesigned by creating tests first. The JUnit tests that were written for MaiBot were created to migrate the functionality over from SimpleBot. These are the steps that were followed: 1. The functionality in SimpleBot was analyzed and broken down into parts that can be verified with a unit test. 2. A JUnit unit test is written. 3. Source code is written for MaiBot to make the unit test from step 2 pass. These steps were repeated for each of the unit tests that were created for MaiBot. The unit tests were provided for the PreFlopModel and PostFlopModel classes first. These classes contain the decision making logic for MaiBot. For the PreFlopModel class, the unit tests provide a summary of the decisions that would be made before the flop. For example, the 31

38 strongest possible pre-flop hand, a pair of aces, would be raised and a unit test verifies that MaiBot makes this decision. The method used to return these actions is preflopaction. Here is the pseudo-code for the preflopaction method (see the MaiBot source code for full implementation details, Used with permission of Biotools, Inc. 2008): public int preflopaction(card cardone, Card cardtwo){ if card 1 rank = card 2 rank and cards are >= 10 or = 2 put in a raise any other pairs, call. if card 1 and card 2 are greater than 10, raise when suited. Unsuited cards, just call. if card 1 and card 2 are suited and difference between card. 1 and card 2 is 1, just call. if card 1 and card 2 are suited and one is an ace and the other is a two, then raise. if one card is an ace and both cards are suited, then call. 32

39 Any other hand should be folded. } The preflopaction class takes the two cards dealt to the player and applies rules to identify the appropriate action. These rules were taken from the SimpleBot source code. Here is an example of a unit test from the PreFlopModelTest class that verifies the action performed when a pair of aces is held pre-flop: public void testpreflopactionaces() { assertequals( Holdem.RAISE, testpfm.preflopaction( new Card(Card.ACE, Card.CLUBS), new Card(Card.ACE, Card.DIAMONDS))); } This test checks if MaiBot's pre-flop decision is to raise if it holds the ace of clubs and ace of diamonds. Additional tests were created to verify the action taken when holding other cards pre-flop. Here is another test: public void testpreflopactionnines() { 33

40 assertequals( Holdem.CALL, testpfm.preflopaction( new Card(Card.NINE, Card.CLUBS), new Card(Card.NINE, Card.DIAMONDS))); } This test checks to see if the action returned is a call when the nine of clubs and the nine of diamonds are held pre-flop. The second set of tests for the PreFlopModel class were a result of a refactoring. Some of the logic implemented in the PreFlopModel class was separated into a smaller method, prefloprandom. The prefloprandom method in the PreFlopModel class provides some logic to randomize the action taken on pre-flop hands that do not fit into the implemented rules. Here is the prefloprandom source code: public int prefloprandom( double amounttocall, double smallbetsize, double callpercentage){ final double CALL_PERCENT = 0.05; // don't fold if there are no bets to call 34

41 if (amounttocall == 0) { return Holdem.CHECK; // play anything 5% of the time } else if (amounttocall <= smallbetsize) { if (callpercentage < CALL_PERCENT) return Holdem.CALL; } return Holdem.FOLD; } This method returns an action that depends on the amount to call, bet size, and a random value to determine the percentage of hands to play. The tests written for the prefloprandom method checked the actual actions returned against expected actions. Here is an example of one of the tests: public void testprefloprandomfold() { assertequals( Holdem.FOLD, testpfm.prefloprandom(5,0,0.05)); } This test checks that the action returned is a fold, according the the parameters provided to the prefloprandom method. 35

42 With the PostFlopModel class, the JUnit tests were created to verify that the action returned by the getpostflopaction method were valid actions for the game. The main goal with the PostFlopModel class was to illustrate the potential use of TDD. The methods in this class were not analyzed fully. Here is an example of a test: public void testpostflopactionjacks() { assertequals( true, testpfm.getpostflopaction( new Card(Card.JACK, Card.CLUBS), new Card(Card.JACK, Card.DIAMONDS)) > 0); } As mentioned earlier, HoldEmAction integrates the functionality of the PreFlopModel and PostFlopModel class. The HoldEmActionTest class provides JUnit tests that verify that the PreFlopModel and PostFlopModel classes were integrated successfully. The JUnit tests implemented in this class are similar to the tests implemented for the PreFlopModel class, which reside in the PreFlopModelTest class. The PostFlopModel functionality was not tested at this time. Tests that are always passing is an important part of the TDD approach to software development. Each test was created incrementally. TDD development on MaiBot strongly supported iterative development. As each test was created, the 36

43 corresponding class was edited to make the test pass. The JUnit framework and eclipse software made it very easy to run all of the tests at once. This feedback verified that any new functionality did not break existing functionality. The tests also serve as documentation for MaiBot. The test names provide enough detail to a developer familiar with poker. Poker robot development is enhanced with TDD. For example, the PreFlopModelTest class verifies the actions that MaiBot takes when holding a certain Hold'Em hand. Strong hands are always played aggressively. Medium hands are always played. Mediocre hands are fed through a method that randomly determines the action. Performance Testing With the MaiBot design completed, along with a set of JUnit tests available, the focus shifted to performance testing. The three JUnit test classes HoldEmActionTest, PreFlopModelTest, and PostFlopModelTest were used to create the performance tests. The performance tests were written in separate test classes. Here is a list of the classes: HoldEmActionPerformanceTests. PostFlopModelPerformanceTests. PreFlopModelPerformanceTests. The performance tests were created using the JUnitPerf framework [19]. These tests measured the amount of time that was required to execute various tests. The time taken to run each test was compared to a maximum allowed time specified in the timed test. If the 37

44 performance tests run under the allowed time, then the test run was considered a pass. Otherwise, the test run was considered a failure. The MaiBot poker robot consists of four classes: MaiBot, HoldEmAction, PreFlopActionModel, and PostFlopActionModel. This poker robot also includes three test classes that demonstrate MaiBot functionality through JUnit tests: HoldEmActionTest, PreFlopActionModelTest, and PostFlopActionModelTest. Finally, three performance test classes measure the execution time of some of the JUnit tests through JUnitPerf testing. The MaiBot design presents some interesting points when compared to the original SimpleBot source code. First, MaiBot consists of a design that displays the decision making in poker. The SimpleBot design consists of only one class, which does not provide much detail by way of modeling. 38

45 Figure 4: Eclipse IDE - Performance Test Run Here is an example performance test from the PreFlopModelPerformanceTests class: public static Test suite() { long maxelapsedtime = 1000; Test testcase = new PreFlopModelTest("testPreFlopActionKings"); Test timedtest = new TimedTest(testCase, maxelapsedtime); } return timedtest; 39

46 This test runs the testpreflopactionkings unit test from the PreFlopModelTest class and measures the time it takes to run this test. This time is compared to the maxelapsedtime value which represents 1000 milliseconds, or 1 second. If the test runs under 1 second, then the test will pass in JUnit. Otherwise, JUnit will display a failure. Here is the test from the PreFlopModelPerformanceTests class: public class PreFlopModelPerformanceTests { public static Test suite() { long maxelapsedtime = 1000; Test testcase = new PreFlopModelTest("testPreFlopActionKings"); Test timedtest = new TimedTest(testCase, maxelapsedtime); } return timedtest; public static void main(string[] args) { junit.textui.testrunner.run(suite()); } } This test runs the testpreflopactionkings JUnit test from the PreFlopActionTest class and checks if it runs under one second. 40

47 Here is the test from the PostFlopModelPerformanceTests class: public class PostFlopModelPerformanceTests { public static Test suite() { long maxelapsedtime = 1000; Test testcase = new PostFlopModelTest("testPostFlopActionAces"); Test timedtest = new TimedTest(testCase, maxelapsedtime); } return timedtest; public static void main(string[] args) { junit.textui.testrunner.run(suite()); } } This test runs the testpostflopactionaces JUnit test from the PostFlopActionTest class and checks if it runs under one second. Finally, the MaiBot program was set up to play poker within the Poker Academy software. Figure 5 shows MaiBot (player Mai) in action against its original version SimpleBot (player Simple). The MaiBot source code was extracted into a Java archive 41

48 (JAR) file. The Meerkat API provides instructions on how to install a poker robot into the Poker Academy environment [17]. Poker Academy was configured to allow the MaiBot program to play against one other poker robot, SimpleBot. Figure 5: MaiBot installed and playing in Poker Academy. SimpleBot was considered to be a working poker robot for Poker Academy. The same assumption was made with MaiBot. The poker game between the two poker robots ran for about an hour and the two poker robots were still playing against each other. The hold'em poker game limits were set to $2 and $4 stakes. SimpleBot started the game with 42

49 $ and MaiBot started with $945. During the hour of play, almost 800 hands were played between the two poker robots. MaiBot showed a win rate of small bets per hand. The SimpleBot and MaiBot decision making algorithms are the same. The MaiBot win rate may be a result of poker variance, especially since only a small sample of hands were played. The play between SimpleBot and MaiBot did not encounter any software errors or crashes. MaiBot demonstrated stability within the Poker Academy environment. The JUnitPerf tests created for MaiBot were executed in the eclipse IDE. The testpreflopactionqueentenhearts test in the HoldEmActionPerformanceTests class took 1-5 milliseconds to fully execute. The testpreflopactionkings test in the PreFlopModelPerformanceTests class also completely executed in the range of 1-5 milliseconds. The testpostflopactionaces test in the PostFlopModelPerfomanceTests class took 146 to 151 milliseconds to execute. The performance tests were executed all at once in the eclipse IDE. The test results show that the pre-flop actions that were tested execute very quickly. This observation is consistent with the rule based pre-flop decision making. The post-flop decision making took considerably longer. The additional computations in post-flop decisions are reflected in the execution time. These results quantify the performance of MaiBot. All of the JUnit tests in the three classes, HoldEmActionTest, PostFlopModelTest, and PreFlopModelTest, successfully executed. The 38 passing tests provide visual 43

50 verification that the MaiBot source code runs as expected. MaiBot provides the same functionality as SimpleBot, with the added benefits of tests to verify functionality and measure performance. Although the MaiBot performance results are expected to be similar to SimpleBot performance, the tests provide MaiBot with the necessary foundation to make attempts to improve its execution time. Poker Robot and Test-Driven Development Discussion The development results for the poker robot MaiBot presents a number of important results. First, TDD leads source code that is developed with a design for test philosophy. Second, the test classes play an important role in verifying the functionality of the source code. Finally, TDD also lends itself well to performance testing. The MaiBot test classes provide a regression test suite that verifies the functionality of the classes and methods. The design of MaiBot presents a basic representation of the decision making required by a Hold'Em poker player. A poker player has a decision to make at the pre-flop stage. After the community cards are dealt, the player then has decisions to make post-flop. These two main decision making stages are modeled in the MaiBot design. The names of the classes, such as PreFlopModel and PostFlopModel, show how MaiBot works. The MaiBot design takes advantage of object oriented programming by using classes that are loosely coupled. Breaking the design into smaller classes made the testing strategy easier to execute. This divide and conquer 44

51 approach made it easier to focus on developing and testing smaller, more manageable pieces of the poker robot. The test classes provide an important purpose in verifying MaiBot functionality. The use of the Eclipse development software made it very easy to run all the tests after changes were made. The short cycles used to write tests, change or add code to MaiBot, and verify that the tests pass, provided useful feedback. This feedback showed previously tested code still behaved as expected after changes were made. This approach to programming can provide an important confidence builder to making changes and additions to the MaiBot source code base. The test classes provide an important start for performance test design and creation. The use of the JUnitPerf test framework provides an important foundation to measure the time it takes the tested MaiBot methods to execute. Since JUnitPerf makes use of the JUnit framework used for MaiBot testing, performance testing became a logical extension of the TDD approach. These performance tests provide basic results that show how long it takes for MaiBot methods to execute. MaiBot's design illustrates one advantage of using TDD. MaiBot displays some features of object oriented design, such as composition of classes. The design for test approach to MaiBot also provides benefits for possible future development. The existing 45

52 tests provide a foundation for other researchers that may be interested in expanding the MaiBot source code. A class diagram for SimpleBot is provided in figure 6. As 46

53 Figure 6: SimpleBot UML Diagram. mentioned earlier, the SimpleBot design consists of only one class. SimpleBot consists of member variables and methods. The member variables deal with different aspects of Hold'Em poker. In a poker hand dealt to SimpleBot, the state is saved in card objects. The game information is also supported as part of Poker Academy integration. For example, the hand evaluator object determines the next decision in a poker hand. SimpleBot takes in game state information from the Poker Academy application so that it makes decisions that follow the rules in Hold'Em. SimpleBot was not developed using TDD practices. SimpleBot does not include any written tests. 47

54 Figure 7 shows a class diagram for MaiBot. This diagram shows the different classes created in the MaiBot design. Figure 7: MaiBot class diagram. Figure 8 shows a class diagram for the tests that were created to test MaiBot. MaiBot provides the same type of functionality as SimpleBot, but the design is composed of more classes. The test classes serve the purpose of verifying the results of calling methods in the MaiBot classes. The TDD approach helped to create confidence in describing the decision making in MaiBot. Pre-flop poker decisions made by MaiBot are 48

55 summarized and verified in the PreFlopModelTest class. The MaiBot decision making can be modified by first adding or modifying an existing test and changing the PreFlopModel class to make the test pass. The entire set of tests for MaiBot also provides regression test values. The tests provide effective feedback to implement changes to the code base. The MaiBot classes show the different decision making stages for a poker player. Like SimpleBot, the game information data is provided to MaiBot, so that it can follow the rules of Hold'Em poker. The PreFlopModel class handles the pre-flop decision making. The PostFlopModel class handles the post-flop decision making. The HoldEmAction class integrates the pre-flop and post-flop decision making. Finally, MaiBot integrates the game information handling with the decision making functionality. 49

56 Figure 8: Class diagram for MaiBot tests. The MaiBot classes present a practical model for a poker player. The decision making is represented, which forms the main actions in a poker game. Although MaiBot and SimpleBot contain the same functionality, MaiBot presents the methods and classes in a model that can be explained in a straightforward manner. 50

57 CHAPTER V RESEARCH IMPLICATIONS Research Results Limitations There are a number of limitations with the results and experience obtained from MaiBot. First, MaiBot is based on an existing, simple poker robot design, which made design changes easier to implement. Second, the testing results are dependent on the hardware and software environment used for development. Third, the test classes do not provide full code coverage for the MaiBot design. The MaiBot design was based on an existing poker robot. Therefore, portions of the MaiBot design was not developed completely in a TDD fashion. Although many of the MaiBot tests were generated after the fact, the MaiBot design still displayed some of the qualities shared with TDD based software projects. This poker robot provided basic decision making for Hold'Em poker playing. MaiBot's poker playing effectiveness was not evaluated. The main objective for MaiBot's design is to illustrate the effects of taking a TDD approach to poker robot development and to show how performance tests can be implemented. The MaiBot design shows how TDD can be used to create a design that takes advantage of objected oriented principles. 51

58 MaiBot development was conducted on a specific computer and software environment, which needs to be considered for further research into this topic. The use of a computer with fewer hardware resources may produce unexpected performance test results. Software compatibility should also be considered to successfully compile and execute the MaiBot source code. The predictability of the MaiBot performance test results depends on careful consideration of the development environment used in this essay. The results for MaiBot testing do not include test code coverage. Source code testing completeness is beyond the scope of this essay. Test code coverage may be a topic of interest in further research. The MaiBot test cases discussed in this essay were created with performance testing in mind. If TDD and performance testing is used in further research, then test code coverage assessment may provide additional value. Specifically, the tests for the PostFlopModel class may be expanded if the methods are understood in greater detail. Rationale for the research The results provided in this essay raise additional items that should be considered in poker robot and general artificial intelligence development. First, TDD and performance testing can go hand in hand, when considering testing strategies for poker robot development. Second, performance testing can be used to help set poker robot 52

59 performance baselines, which can lead to improved performance. Third, performance testing can benefit artificial intelligence software projects, in general. MaiBot development provides positive results with regards to TDD and performance testing. The source code developed presents a design that lends itself well to testing. The systematic process in TDD development facilitates performance test case creation. TDD and performance testing provides a method to isolate certain parts of the MaiBot source code. The performance tests provide the necessary information to evaluate the time it takes to perform a JUnit test. Since the JUnit tests verify the functionality of the MaiBot source code, performance tests verify the time it takes to execute the code. The use of TDD and performance testing can help poker robot researchers and developers improve their robot performance. The performance tests created for the MaiBot source code provides a performance baseline. The time thresholds provided in the performance test cases demonstrate the current performance of the MaiBot program. If changes are made to the program, the performance tests provide the necessary feedback to evaluate the impact on execution time. In poker, decisions should be made within a reasonable amount of time. Poker robot researchers can define parameters that meet their defined time thresholds for decision making. These time thresholds can then be programmed into performance test cases, like 53

60 MaiBot. The MaiBot results show the positive impact that tests cases have in evaluating poker robot performance. The MaiBot JUnit and JUnitPerf tests were developed and executed with the following hardware and software: Laptop: Acer Aspire 5570 Processor: Intel(R) Core(TM) Duo CPU 2.00GHz Memory: 1.00 GB Operating System: Windows Vista(TM) Home Premium (32-bit) Java Development: eclipse Ganymede (version 3.4) Java Runtime Environment: Version JUnit Version JUnitPerf Version 1.9 The results obtained in this essay may be applied to other software based artificial intelligence projects. Performance testing can be very useful in projects where there are important execution time goals. Well defined testing strategies using TDD and performance testing can show researchers if they can meet their performance goals, or if additional changes are necessary. With performance tests, researchers have meaningful tools and data so that they can measure the impact of program changes. MaiBot development demonstrated a practical application of software programming techniques. 54

61 TDD appears to be a combination of existing software development concepts. The application of TDD provides a useful set of tests that serve as a tool that evolves with the design of the poker robot. Future developments in the field of poker robots can be improved with the use of TDD. Software algorithms can be developed with testing in mind. This testing helps with poker robot verification. TDD may form a valuable part of a poker robot developer's tool set. There are some interesting opportunities for further research in poker robot research, TDD, and performance testing. Researchers may investigate the benefits of using TDD to improve poker robot design and algorithms. Combining the research on poker robot development and TDD may provide the computer science community with an organized method of examining performance. Future progress in these fields may benefit by teaching TDD in undergraduate computer science classes. The introduction of TDD to course work has provided instructors with an effective learning tool. This exposure to TDD may encourage future poker robot developers to apply TDD and performance testing to their work. Further development is also possible with MaiBot. First, the algorithms implemented may be improved. With a foundation of tests to build on, changes to the source code may not be too difficult. Second, the set of tests may be expanded to provide greater test coverage and understanding of the implementation. Since this essay provided 55

62 a basic look at the application of TDD and performance testing to poker robot development, additional tests can be written for the MaiBot source code. Finally, the results obtained with MaiBot may complement the progress made in other research areas. The potential uses of TDD and performance testing are not limited to poker robot development. 56

63 CHAPTER VI CONCLUSIONS The experience from MaiBot development provide important results. The tests that were created for MaiBot provide additional feedback and verification on the method. The TDD development process resulted in functionality verification and the application of object oriented software design. The MaiBot test classes provided 38 tests that run and verify methods that make poker playing decisions. The MaiBot performance tests displayed the quick pre-flop decision making and the more computationally intensive post-flop decision making. MaiBot was successfully integrated with the Poker Academy software and successfully played a number of Hold'Em poker hands with another robot, SimpleBot. The tests form a valuable tool to provide effective feedback on MaiBot functionality. The use of tests to verify poker robot decision making was an important part of MaiBot development. The tests provided documentation on the types of decisions that MaiBot made with specific hands. The performance tests measure the time it took MaiBot to make some poker hand decision making. These set of tests provide the foundation for further improvements on the algorithms and performance of MaiBot. Although MaiBot is specifically designed to play poker, the TDD development approach can be successfully applied to other fields involving software development. 57

64 The results of MaiBot development provide a practical application of the theory behind poker robots, TDD, and performance testing. Poker robot researchers may find TDD to be a useful development design tool. Performance testing may also be very useful due to the time sensitive nature of poker playing. The benefits in using TDD and performance testing are evident in the MaiBot design. The MaiBot tests can reduce the number of defects and the resulting source code exhibits object oriented principles. MaiBot was developed using eclipse. The eclipse software is an excellent tool for TDD development with the Java programming language. There are some limitations with the MaiBot results. The source code was based on an existing poker robot that did not include any tests. The functionality was not altered. The main objectives with MaiBot development were to create tests to verify functionality and measure test performance. The MaiBot source code was not modified to improve performance. Further research can be conducted to evaluate the benefits of using MaiBot and its set of tests to improve poker playing performance. The set of tests included with MaiBot do not provide full test code coverage. Further code coverage may provide additional benefits to reduce potential defects. In conclusion, TDD development can be applied successfully to the field of poker robots. The ability to create a set of tests for a poker robot provides important verification 58

65 results. Performance testing provides additional feedback to measure execution time. The use of performance testing can help researchers improve their software. 59

66 REFERENCES [1] L. Greenemeier, "Poker-Playing Robots Battle For $100,000 Pot," InformationWeek, [On-line document], 2005, [cited 2008 Jan 15], Available HTTP: [2] E. M. Burke and B. M. Coyner, Java Extreme Programming Cookbook, 1st ed., Sebastopol: O'Reilly, [3] R. Mugridge, Test driven development and the scientific method, Proceedings of the Agile Development Conference, pp , [4] University of Alberta Computer Poker Research Group, 2007, [cited 2007 Nov 3], Available HTTP: [5] D. Billings, N. Burch, A. Davidson, R. Holte, J. Schaeffer, T. Schauenberg, and D. Szafron, "Approximating Game-Theoretic Optimal Strategies for Full-scale Poker,"[On-line document], 2003, [cited 2007 Nov 3], Available HTTP: [6] D. Billings, D. Papp, J. Schaeffer, and D. Szafron, Opponent Modeling in Poker, [On-line document], 1998, [cited 2007 Nov 3], Available HTTP: [7] D. Billings, D. Papp, L. Pena, J. Schaeffer, and D. Szafron, "Using Selective- Sampling Simulations in Poker," 1999, [cited 2007 Nov 29], Available HTTP: [8] C. Wilson, " Raise you 50..." New Scientist, vol 180, issue , [9] L. Williams, E. M. Maximilien and M. Vouk, Test-driven development as a defectreduction practice, 14th International Symposium on Software Reliability Engineering, pp , ISSRE [10] H. Erdogmus, M. Morisio and M. Torchiano, On the effectiveness of the test-first approach to programming, IEEE Transactions on Software Engineering, vol. 31 (3): , [11] G. Saurer, J. Schiefer, and A. Schatten, Testing complex business process solutions, The First International Conference on Availability, Reliability and Security, (20-22):8,

67 [12] L. Crispin, Driving Software Quality: How Test-Driven Development Impacts Software Quality, IEEE Software, vol. 23 (6):pp , [13] T. Karamat and A.N. Jamil, Reducing Test Cost and Improving Documentation In TDD (Test Driven Development), Seventh ACIS International Conference on Software Engineering, Artificial Intelligence, Networking, and Parallel/Distributed Computing, (19-20):73-76, SNPD [14] R.C. Martin, Professionalism and Test-Driven Development, IEEE Software, vol. 24 (3):32-36, [15] J. Miller and M. Smith, A TDD approach to introducing students to embedded programming, Proceedings of the 12th annual SIGCSE conference on Innovation and technology in computer science education, Dundee, Scotland, pp , [16] T. Briggs and C.D. Girard, Tools and techniques for test-driven learning in CS1, Journal of Computing Sciences in Colleges, vol.22(3):37-43, [17] "Poker Academy," [On-line document], 2005 [cited 2007 Nov 29], Available HTTP: [18] JUnit, Testing Resources for Extreme Programming, 2008; [cited 2008 Feb 26], Available HTTP: [19] JUnitPerf, 2006, [cited 2008 Feb 26], Available HTTP: [20] eclipse, An open development platform, 2008, [cited 2008 Feb 26], Available HTTP: [21] D. Billings, Computer poker overview, University of Alberta, [22] D. Billings, Algorithms and Assessment in Computer Poker, PhD Thesis, University of Alberta, [23] D. Billings, A. Davidson, J. Schaeffer, and D. Szafron, "Improved Opponent Modeling in Poker," 2000, [cited 2005 June 29], Available HTTP: [24] T.C. Schauenberg, Opponent modeling and counter-strategy for poker, Master's thesis,

68 APPENDIX A PROGRAM DOCUMENTATION RUNNING PROJECT TESTS Two items are required to execute the JUnit and JUnitPerf tests for the MaiBot program: 1. Eclipse IDE software for Java. 2. MaiBot project files. First, the Eclipse software should be downloaded and extracted to the local PC hard drive. The Eclipse web-site is Next, the Eclipse software is downloaded in.zip format. It must be extracted. Once the software is extracted, the Eclipse program can be used. This completes the Eclipse software installation instructions. Now the MaiBot project source code can be imported into the Eclipse environment. From the Eclipse menu, select File > Import. 62

69 Figure 9: Import select screen. From the Import screen, the General section should be expanded and the Existing Projects into Workspace should be selected. Press the Next button (See Figure)On the Select root directory section, select the directory for the MaiBot source code. Poker should be displayed in the Projects section. Press the Finish button to import the Poker project. The project should now be available in the Eclipse environment (see Figure 4). All of the tests can now be executed in the Eclipse environment. First, verify that the Poker project is selected in Eclipse. Then all of the tests (JUnit and JUnitPerf) can be executed by selecting Run > Run As > JUnit Test. The following results should be 63

70 shown in the JUnit section of the Eclipse environment. When the Console section is selected the JUnitPerf test execution times are shown. Figure 10: Import project screen. 64