Parameter-Free Tree Style Pipeline in Asynchronous Parallel Game-Tree Search

Transcription

1 Parameter-Free Tree Style Pipeline in Asynchronous Parallel Game-Tree Search Shu YOKOYAMA, Tomoyuki KANEKO, Tetsuro TANAKA T11:15+02:00 ACG2015 Leiden

2 Motivation Game tree search in distributed computing Target env.: resources via slow network Constraint: limited communication high latency, narrow bandwidth Too much: sharing of transposition table Common, Cheap (e.g. Ethernet LAN) e.g., TDSAB, APHID: not practical in this environment GPS shogi: is successful example 2

3 P GPP: Parameter Free GPS shogi GPS shogi: Our team s program (2013) Defeated professional Connecting 679 imacs GPP: Game Position Parallelization Photo of the match GPS shogi vs. Mr. Miura (see ) 3

4 P GPP: Parameter Free GPS shogi GPS shogi: Our team s program (2013) Defeated professional Connecting 679 imacs GPP: Game Position Parallelization New Method: GPP with Parameter Free Adjustment by hand Stand alone program Expert s game records Our method Distributed search system 4

5 GPP overview Master Worker system Master (server): Constructs a master tree Worker (client): Search independently root (using shared memory parall. methods) d4 e4 Nf6 e6 Worker C e5 Worker E Worker D Worker A Worker B Worker F A leaf of the master tree = A Worker 5

6 Communication in GPP (1) Reconstruct the master tree Allocate positions to workers Search by workers Worker 2, Your duty is Pos. A. Master Make a move (by system or opponent) current pos. The master tree Worker 1 Worker 2 Worker 3 Pos. A Pos. B?? Eval. Value:???? 6

7 Communication in GPP (2) Reconstruct the master tree Allocate positions to workers Search by workers Make a move (by system or opponent) Periodical Report current pos. Master The master tree Worker 1 Worker 2 Worker 3 Pos. A Pos. B?? Eval. value of Pos. A is 30 Eval. Value 30?? Refresh 7

8 Reconstruct the master tree Allocate positions to workers Search by workers Make a move (by system or opponent) Tree Expansion Tree Style Pipeline Method Worker Pool Played pos. 8

9 Reconstruct the master tree Allocate positions to workers Search by workers Make a move (by system or opponent) Tree Expansion Tree Style Pipeline Method Worker Pool Played pos. 9

10 Reconstruct the master tree Allocate positions to workers Search by workers Make a move (by system or opponent) Tree Expansion Tree Style Pipeline Method Worker Pool Played pos. 10

11 Our Method New Strategy of Tree Growing GPS shogi: many heuristic parameters for tree expansion Fully automated tuning of parameters Tree expansion w.r.t. utility of master tree Application to chess 11

12 Goal of Worker Managements Minimization Maximization Search efficiency Model: estimation by realization probability 12

13 Tree Expansion and Realization Probability Realization Probability for each leaf [17] (Tsuruoka, 2002) root DEEPEN root WIDEN root A A 0.45 A B α α (0.45) (0.55) Expanding a leaf Dividing the realization prob. 13

14 Estimation of Realization Probability The worker evaluates the expert s move as rank n n p n p 1 (1- p 1 - p 2 ) = second best after root s bestmove p 1 p 2 =

15 Definition: Master Tree s Utility Current Pos. Avg. time of Real Tree Ignoring edge to others for counting depth 15

16 Tree Expansion and Utility Change 0.45 root DEEPEN 0.45 WIDEN 0.30 A 0.55 α 0.35 A 0.20 α 0.35 A 0.20 B ( Increase of Σ ) = (new leaf s realization prob.) can be maximized by GREEDY algo. 16

17 Greedy Expansion of Tree DEEPEN A DEEPEN B WIDEN WIDEN C D A 0.23 B 0.05 C 0.12 D 0.16 Which candidate leaf will have the biggest prob.? Add Repeat until all workers are assigned A to tree 17

18 In Practice: Local Tree Extraction from TT Greedy algo. says: Only worker knows <rank, move> Nd3 g3 f4 g3 f4 Nd3 GPS shogi used preliminary search in 1s 18

19 Self Play Experiments How playing strength improved? Implementation Master: TCP Server, C++ with boost::asio Worker: Stockfish DD + minor changes (single thread search) Self play: Sequential vs. Parallel Ran on 64 equivalent cores (32 cores * 2PCs) Fixed thinking time (1000ms per ply) 19

20 Time Control and Penalty Thinking Time: Fixed To simplify the program and exclude environ. differences Our Workers Opponent [ms] think ponder t Thinking Time Penalty: 50ms Suffiient cost for Extraction from hash table on worker Sending information via network Expansion of master tree

21 Results Win Rate [9] Optimistic Pondering [9] (Himstedt, 2012) Time: 20min +5s Our Program Time: 0 +1s Due to the time penalty # Workers 60 Very short hard to parallelize 21

22 Conclusion and Future Work Conclusion: Our parallel search effectively worked in short time matches with high latency network (preferable in distributed computing) without hand tuned parameters (preferable in application to many games) Future Works Improving the estimation of realization probability Experiments with 128 nodes 22