Augmented Reality Chess Assistance. Matthew Berntson

Transcription

1 Augmented Reality Chess Assistance Matthew Berntson

2 Goal Service for chess players to review past games, move by move, offline. Work for screen captures of chess games in digital format. Saved classic games. Incorporate a chess playing bot to find an optimal move.

3 Assumptions Accept if 16 pieces are on the board in the correct starting positions, regardless of organization. The background of the board is lighter than its border. The previous chess move was legal. Travel chess piece style.

4 Previous Work Identifying corners of checkerboard normxcorr2 Normalized 2D cross correlation SIFT Chess Programming Wiki Repository of chess playing engines, such as the site s own CPW Engine and Glass.

5 Technical Approach Intended to detect chess board using the findcheckerboard code. Inconsistent identification, often finds sub rectangles of the board. Intended to detect piece types from one frame to another Inconsistent matching and feature detection

6 SIFT Failure

7 Technical Approach Picture is taken such that the board takes up almost the entire image. Determine the edges of the board by finding gradients from the edge of the image. Grayscale image necessitates varying gradient thresholds to detect board border.

8 Technical Approach Solution: work with binary image. Threshold the image with a low gray threshold in order to ensure that the background is white in the BW image. Use a depth threshold of 8 10 px for border detections to avoid false detections of noise.

9 Technical Approach Take 2 points of detection for that edge. Find linear model for each edge. Use perpendicular line intersect to find the corner of two edges One line is independent in x, the other in y.

10 Technical Approach Game State Legitimate setup board Turn player Check Lost King No moves are prevented, so the turn player can move into check; lose.

11 Processing Given corners: transform to orthophoto and partition into 64 squares. For setup frame: check correct board orientation and that the middle squares are unoccupied. For later frames: subtract current square from previous square to determine which two squares have been altered. For all frames: calculate and display all possible moves.

12 Processing Experiments Results Sum over unoccupied square: black<1000, white>9000. Sum over occupied square: black<5000, white>=5000. Magnitude of sum over difference square: move>100, no move<=100. Using a higher threshold for move could insure against false detections

13 Processing by summing pixels Sums: From= 837, To= 3989, wk= 83

14 Processing by summing pixels Sums: From= 2350, To= 3247, Error (br)= 29

15 Processing by summing pixels Sums: From= 2252, To= 2046

16 Processing by summing pixels Sum of Empty (4,4)= 9512

17 Processing by summing pixels Sums on take w/ wq: From= 2006, To= 159

18 Limitations Data are dependent on the area of white paper background for the piece; on the cutting method. This problem is avoided when using digital inputs. Does not handle multiple pieces moving. Castling would be an extension. Moving a piece within its square could cause a false detection. Thresholding seems to avoid this problem.

19 Future Work Compare detection success to implementations published for identifying features in binary images. Port board configuration to python engines provided by Chess Programming Wiki in order to calculate and augment with an optimal move for the turn player. Extract more advanced game state information, such as castling, promotion, check, pruning moves that would put the turn player in check, checkmate.

20 Detected Board