Informatica Universiteit van Amsterdam

Transcription

1 Bachelor Informatica Informatica Universiteit van Amsterdam Designing an unobtrusive chess humanmachine interface Esan Wit, 6333 August 9, 22 Supervisors: Robert Belleman (UvA) Signed:

2 II

3 Abstract This thesis discusses the use of an RGB-D sensor, the Microsoft Kinect, to accomplish classification of chess pieces. It illustrates the use of depth profiles for the accurate identification and differentiation of objects on the chess board. Various experiments illustrate the abilities of the depth profile and the possible setups in which the Kinect can be used to observe the chessboard. It shows that the Kinect is capable, even at the edge of its range, of attaining enough data to construct a unique descriptor for the pieces. It also shows that depth profiles are sufficiently rotational invariant for classification purposes with simple models. Finally several recommendations are made for future research to expand upon.

4 Contents Introduction. Context Goal Related Work 3 3 System design 4 3. Chessboard Kinect libfreenect Chess interface Methods 7 4. Depth mapping Segmentation Histogram matching Model learning Classification Setup calibration Experiments 5. Testing the hypothesis Results with the Kinect at a height of 6cm Results with the Kinect at a height of 92cm Capturing the board Four sensors, quarter board Setup Results Black pieces Two Sensors, half board Setup Results Conclusions 29 7 Future Work 3 II

5 CHAPTER Introduction. Context Traditionally RGB systems or dedicated hardware and special markers are used to enable computers to play chess with a real pieces. For instance systems using markers on the top or bottom of chess pieces and a sensor to detect those markers or simply recognizing the outline of chess pieces against a background[5]. However both of these solutions come with a problem namely they either required dedicated, marked, pieces or they are sensitive to occlusion. An example of the occlusion could be a pawn hidden behind a king from the camera s point of view. Also the use of an RGB camera comes with a shortcoming within the domain of chess, namely that the pieces are predominantly black and white this means that a great amount of information usually available in the visible domain is unavailable on a chessboard..2 Goal This research aims to design a non-obtrusive system that can monitor real world chess pieces to determine moves made in the game. The goal is to device a system that can register the playing field and recognize if pieces have moved and what piece is located before it, however it should not hinder the user during gameplay by requiring active input. This introduces several constraints on the system:. The system should be unobtrusive and not require user input during play. 2. The system should operate regardless of how pieces are placed on the board and when the system is activated. 3. The system should be cheap to implement and easy to set up. 4. The system should provide a robust method of classification. These constraints lead to the following purpose of this research: Can an automated system be created that is unobtrusive, robust, cheap, and easy to set up which can enable the computer to use a real chessboard as an input device?

6 In order to achieve this goal we propose a system based around the Microsoft Kinect 2, the Kinect is capable of using both an RGB and a depth camera. The expectation is that the depth view of a chessboard offers enough information to identify pieces placed on the board. The use of the Kinect also allows for placement of the sensor above the scene making it virtually impossible for an object to block the scene from the camera aside from when a move is made. The hypothesis is that the Kinect offers a solution to the majority of the constraints. The Kinect will function as the eyes of the system identifying the pieces on the board, a chess engine can then use this information to determine the next move to play, lastly a monitor can be used to visualize this move to the user

7 CHAPTER 2 Related Work Related work in this area is often focused on using a single camera to capture the field and determine the moves made. Some methods work with a single camera on the board and partially determine moves by comparing images from two situations, one before and one after image, and by using the difference image determine the pieces moved and affected[3]. However as Blunsden[2] remarks in his work there is an inherit downside in this method in that it may fail to register moves such as taking opposing pieces which might only change a single visible square. Other solutions propose the use of a secondary camera to help determine moves and get around the problem of partially obscured pieces[4, 5] by looking from a different angle and rely on known piece configurations to aid in the classification of pieces on the board. However this just shifts the problem and does not make it less likely that the secondary view may be obstructed as well. It is for this reason that the system proposed in this thesis is using a top-down view of the board. 3

8 CHAPTER 3 System design 3. Chessboard The system is designed to work with a specific chess set, a large set, located in the hall of the Faculty of Science building of the University of Amsterdam at Sciencepark. The chess set has the following specifics (see also Figure 3.): Playing Field Width:3.6m Length:3.6m 2 Kings Height: 63cm 2 Queens Height: 58cm 4 Bishops Height: 56cm 4 Knights Height: 48cm 4 Rooks Height: 43cm 6 Pawns Height: 43cm The chess pieces are made from plastic and are black or white in color, the black pieces having a polished finish and the white pieces having a matte finish. 3.2 Kinect The Kinect is a RGB-D sensor, it captures RGB and depth images of the environment, developed by Microsoft and is used primarily for use with the Xbox 36 console. However due to the availability of drivers it is also possible to use the Kinect on other systems. The Kinect has a 58 horizontal and 43 vertical field of view[6, ] and is capable of capturing up to 3 frames per second for both the RGB and depth sensor[]. The Kinect is most effective in a range of meters according to specifications[6], however the sensor provides depth values even outside the optimum range, albeit less precise and more prone to error. Smit also touches upon several problems that exist with the Kinect. Firstly the fact that the raw data delivered by the Kinect suffers from jitter, meaning the variation of recorded depth in a static environment. For each frame recorded there is a variation that could result in values being off by an amount of 4 or less. The second problem is 4

9 (a) The white pieces with a mostly matte finish. (b) The black pieces with a polished finish. Figure 3.: The pieces for which the system is to be designed. that the accuracy of the depth values decreases as the distance between the Kinect and the scene increases[]. When observing a static scene the Kinect does not transmit the same depth value, unfortunately there is some jitter on the depth axis, to reduce the effect of this an average image is calculated over 5 samples to reduce noise. All calculations in this thesis are performed on averaged images over 5 samples, unless stated differently. In effect this would cause a.67 second delay before the scene can be processed by the system in order to acquire the required samples libfreenect In order to communicate with the Kinect sensor the libfreenect library is used. Libfreenect enables a non-xbox 36 system to communicate with the device, and offers direct access to the raw data retrieved from the sensor. Since the program and algorithms used are written in Python the libfreenect Python wrapper is used in combination with numpy2. The libfreenect library offers the data to the system using a 64 by 48 numpy array containing bit values. The values signify a distance to the object a lower value being closer to the Kinect then a high value, a value of 247 is used to signify a point where no distance could be determined. 3.3 Chess interface For the task of identifying the chess pieces a system based on the Kinect is proposed. The Kinect gathers input from the real world and can be analyzed two-fold. Firstly the Kinect detects when a player is moving over the field this is determined to be the moving phase, when the player has left the area designated for the playing field the system will enter the detection phase and analyze the field to determine the move made, or if it was just a random movement over the field. If a move was made the system will feed the current board situation to the chess engine, the thinking phase, after the chess

10 engine determines a move, or signals an invalid move was made, or the game has ended, the system visualizes the move by means of a screen a wall by the chessboard so the player can move the piece and acts similarly to the moving phase. After the player has moved the piece shown by the engine the system resets back and awaits the next move. This design describes neither the amount of Kinects required nor how the detection will be done since both are dependent on the exact situation and the preferred results. If a single Kinect is unable to gather enough data of the board then two or more may be used to improve results. 6

11 CHAPTER 4 Methods 4. Depth mapping In order to identify the pieces on the chessboard the distance histogram to the pieces is used to define the profile of the different pieces. This distance histogram is created using the depth image provided by the libfreenect library. From this image a single square is extracted, this square is then used to construct a histogram of all values within a certain range, this range is determined using the ground level, determined from an empty scene of the board, and the closest point of the king, the king being the highest piece on the board, this ensures that the values that describe other pieces are within these two limits. The histograms retrieved from this are normalized to account for the difference in the amount of values left in the described range, this adds a side effect that a slight difference in placement of the pieces does not result in the misclassification due to the size of the input frame. Histograms are used because they introduce an independence from the rotation of the object in the image. These histograms are later used by the classifier. It is possible to use only a single model regardless of the square in question however this may have lesser results, due to ignoring the possible skew that occurs depending on the location of the piece, and is therefore not recommended. It is recommended to create different models for each location on the board. Multiple models of the same piece may be learned for improved accuracy and robustness. 4.2 Segmentation In order to locate pieces on the board the image is segmented into squares according to board positions. This eliminates the need to find the squares in each frame but also forces the system to require the exact location of each square during setup. During segmentation it should be taken into account that the squares used in the depth frame are dependent on the height of the pieces and must be compensated, hence the squares selected for the depth image are slightly larger then those used by the board. 7

12 4.3 Histogram matching To compare the histograms belonging to a known and unknown piece a similarity score is determined between the two histograms. The similarity between the histograms is defined as follows: S(H, H 2 ) = + I H I H 2I (4.) Where H and H 2 are histograms between which to calculate a similarity. Note that this similarity is only defined over normalized histograms. Sum(H) = Also note that the minimum similarity between two normalized histograms that do not match is given as 3 because the greatest absolute distance between two normalized histograms equals 2. The model that has the greatest similarity to the piece is used to classify the unknown piece. 4.4 Model learning All comparisons for the identification steps are done against a reference model. Determining the reference model is intended to be a simple learning step. For each segmented square a model is built of the various pieces. This model is based on the depth profile histogram of the segmented squares. For each square a model is made of what the pieces will look like on that square. Due to symmetry of the board and pieces it depends on the amount of sensors used for how many models have to be build. If a single camera is used there would be a requirement of 6 models. When the amount of cameras doubles the required amount of models halves. However because the board is square the suggested amounts of cameras are powers of Classification The classification of pieces is done by determining the difference between the reference model and the unknown piece in the square and relies heavily on the uniqueness of the depth profile of pieces. The classifier uses the similarity function described in 4.3 to compare input to the model. The class that shows greatest similarity with the unknown is used. The classifier in this thesis functions using a single representation of a chess piece per square. This eliminates the need for rigorous learning before use but possibly makes the system susceptible to misclassifying rotated pieces. It is possible to extend the classifier to use a range of models per piece rather than a single model. = N i= H i 8

13 4.6 Setup calibration Setup calibration is the initial information required by the system to operate. In order to develop the reference model that is used for identification the system must know the position of the board in respect to the data received; this is to be configured by the end user after the final setup is known. During setup the playing field is defined to the system after which the squares are determined for the segmentation process. The reference model also needs to be introduced to the system. This means that the user should submit samples of pieces on pre-designated positions on the field. In an optimal situation, due to symmetry of the board, only a quarter of the field needs to be defined during this setup. 9

14 CHAPTER 5 Experiments 5. Testing the hypothesis In order to test if the Kinect can be used for observing the chessboard, with the classifier as proposed in 4.5, a test setup was built with the sensor directly above the pieces looking straight down. Due to the depth measure returned from the sensor being distributed over a logarithmic scale, as described in 3.2 this is measured for different heights. In all these tests the pieces are placed in known rotations directly under the sensor. In order to verify wether or not the pieces are truly rotational symmetrical in their length axis multiple angles are recorded. For most pieces merely 3 rotations are tested due to expected symmetry in the resulting image. Other asymmetrical pieces are recorded more often until a measure of symmetry is reached, exact details can be found in Table 5.. The images used for the reference model are the average over 5 samples, this is done to eliminate jitter. With the 3 frames per second delivered by the sensor this equals 2 3seconds of data. 5.. Results with the Kinect at a height of 6cm In order to test the symmetrical integrity of the pieces a comparison must be made between the different rotations available to compare the effect the rotation has on the object. In Figure 5. a visual representation is shown, note that the image is histogram equalized in order to increase visual differences between the levels in the image. Due to problems with obtaining consistent imagery for black pieces those are treated separately King(k) Queen(q) Bishop(b) Knight(n) Rook(r) Pawn(p) Table 5.: The rotations of the different pieces from the camera s point of view.

15 (a) King (b) Queen (c) Bishop (d) Knight (e) Rook (f) Pawn Figure 5.: Depth images of the pieces from a height of 6cm (black is distant, white is close). later on. The histograms in Figure 5.2 are the reference model used for the classifier. The model shows clear differences between the various chess pieces. However due to the use of an unchanged depth value of the sensor the model is very sensitive to the location of the piece, if the piece is moved from the exact spot beneath the sensor the model created from this will change. Little changes become shifts of the values over the x axis in Figure 5.2, greater changes can shift the model as far as that there is almost no remaining similarity between the different pieces. It is therefore pertinent that the pieces are centered in the square. The first step in testing the hypotheses is done by checking if the calculated histograms are indeed rotationally similar. In Figure 5.3 it can be seen that the items show remarkable rotational similarity. It seems that only the king and queen seem to show any resemblance to each other, comparison between different types generally score a similarity of except for the king and queen where a similarity of up to.63 appears. Minor differences between the various rotations are contributed to a slight off center position of the piece or to the fact that the sensor projects points from one side and records visible points from another side of the sensor. This also results in the black spots recorded on the images of the bishop, Figure 5.c, and pawn, Figure 5.f. Black pieces As noted earlier the black pieces offer difficulties in being detected by the sensor, this is most likely due to the glossy coating of the black pieces. When the pieces are compared at this height against the model build with the white pieces the result however is accurate enough to identify all pieces correctly, although its likely to be more prone to wrongfully identifying pieces that have shifted from the center of the square, see Figure 5.4a. When a separate model is build solely for the purpose of classifying the black pieces results are vastly improved as can be seen in Figure 5.4b.

16 B B B K.5569 B K K NB.. K N N Q N Q K Q.. P.. Q N P PR Q.. P R R P R R model value model King model Queen model Queen model King model Queen model Queen model King model King model Queen model King model Knight 772 model Bishop 836 model (a) 772 Depth King 794 value 85 (b) Queen Depth value Depth 85 value Bishop Depth Depth model value value Knight Depth model value Bishop model Knight model Bishop model Knight model Bishop model Knight model.25 King model Knight mode value odel Pawn model Rook model Rook Depth model value Pawn Depth Depth model value value (c) Bishop Rook model (d) Pawn Knight model Rook model Pawn model Pawn model value Depth value (e) Rook (f) Pawn Figure 5.2: Histograms created from the pieces shown in Figure 5.. Histograms made using the sample data over the averaged image of 5 frames. 2

17 TopDown/H6 B B TopDown/H6 B45 B TopDown/H6 B9 B TopDown/H6 K K TopDown/H6 K45 K TopDown/H6 K9 K TopDown/H6 K35 K TopDown/H6 N N TopDown/H6 N45 N TopDown/H6 N9 N TopDown/H6 N35 N TopDown/H6 N8 N TopDown/H6 N225 N TopDown/H6 N27 N TopDown/H6 N35 N TopDown/H6 P P TopDown/H6 P45 P TopDown/H6 P9 P TopDown/H6 Q Q TopDown/H6 Q45 Q TopDown/H6 Q9 Q TopDown/H6 R R TopDown/H6 R45 R TopDown/H6 R9 R Type evaluation Similarity Ground B B45 B9 K K45 K9 K35 N N45 N9 N35 N8 N225 N27 N35 P P45 P9 Q Q45 Q9 R R45 R9 K Q B N R P Sample Figure 5.3: Rotational symmetry from a top-down perspective between chess pieces at a height of 6cm. The bars on the x axis are grouped by 6 and each bar signals a single piece being compared against one of the reference models; the label on the axis describes the sample piece that is being compared. The bars, within each group of 6, are ordered from left to right as follows: King(k), Queen(q), Bishop(b), Knight(n), Rook(r), Pawn(p). The y axis shows the similarity score of a piece against the model. 3

18 TopDown/H6 P45 P TopDown/H6 P9 P TopDown/H6 Datafile Q Class Q Result K Q B N R P TopDown/H6 Q45 Ground K Q TopDown/H6 B Q9 B Q TopDown/H6 B45 R B R TopDown/H6 B9 R45 B R TopDown/H6 K R9 K R TopDown/H6 K45 K TopDown/H6 K9 K Type evaluation TopDown/H6 K35 K TopDown/H6 N N TopDown/H6 N45 N TopDown/H6 N9 N Black pieces against black model TopDown/H6 N35 N h6. TopDown/H6 N8 N TopDown/H6.625 N225 N TopDown/H6 N27 N TopDown/H6 N35 N TopDown/H6 P P TopDown/H6 P45 P TopDown/H6.25 P9 P TopDown/H6 Q Q TopDown/H6.25 Q45 Q TopDown/H6 Q9 Q TopDown/H6 Ground R B B45 R B9 K K45 K K35 N N N9 N N8 N225 N27. N P P45 P9 Q Q45 Q9 R R45 R9 TopDown/H6 R45 R K Q B N R P Sample TopDown/H6 R9 R Similarity Type evaluation Similarity Ground B B45 B9 K K45 K9 K35 N N45 N9 N35 N8 N225 N27 N35 P P45 P9 Q Q45 Q9 R R45 R9 K Q B N R P Sample (a) Classification of black pieces using the white model. (b) Classification of black pieces using the black model. Figure 5.4: Classification of black pieces 4

19 (a) King (b) Queen (c) Bishop (d) Knight (e) Rook (f) Pawn Figure 5.5: Pieces from a height of 92cm 5..2 Results with the Kinect at a height of 92cm The experiment is also conducted on a height of 92cm from ground level to determine the difference that distance between object and camera introduces to the system. from a graphical representation the pieces have hardly changed as can be seen in Figure 5.5. When the models are compared between the heights of 6cm and 92cm there is once again very little difference. In Figure 5.6 the model is shown for a height of 92cm. When compared against the model in Figure 5.2 not many differences are seen aside from the shifting of the values on the x-axis. The histograms probably compensate for the differences of the logarithmic depth values by dividing the points in their respective bins, at greater distances between piece and camera these models should change significantly. When comparing the results for the classifier, as shown in Figure 5.7, it is note worthy that results have improved as the distance increased. The similarity between different pieces are stuck below with the exception of the king and queen which are below 5 which is less then the similarity scored by king and queen at a height of 6cm which resulted in a similarity of around.63. Black pieces The black pieces at a height of 92cm show a similar reduced visibility as at a height of 6 cm. However due to the increase in height there is a decreased effect of off center placement in the square on the classification process. This results in similar results as long as a separate model is learned for the black pieces. 5

20 .. B. B N K K Q B N. N P K Q. Q R N P P Q R R P R el King model Queen model Queen model Queen model Queen model King model King model King model King el (a) Knight King 86 86model Depth value Bishop model Bishop model Bishop model (b) Queen Depth value Bishop Depth value model Knight model Knight model Knight model Dep Knig l Pawn model (c) Rook Bishop model Rook model Rook model Depth valuedepth value Rook model Pawn (d) model Pawn Knight model Pawn model Dep Paw Depth valuedepth value Depth valuedepth value Dep (e) Rook (f) Pawn Figure 5.6: Histograms created from the pieces shown in Figure

21 TopDown/H6 K9 K TopDown/H6 K35 K TopDown/H6 N N TopDown/H6 N45 N TopDown/H6 N9 N TopDown/H6 N35 N TopDown/H6 N8 N TopDown/H6 N225 N TopDown/H6 N27 N TopDown/H6 N35 N TopDown/H6 P P TopDown/H6 P45 P TopDown/H6 P9 P TopDown/H6 Q Q TopDown/H6 Q45 Q TopDown/H6 Q9 Q TopDown/H6 R R TopDown/H6 R45 R TopDown/H6 R9 R Type evaluation Similarity Ground B B45 B9 K K45 K9 K35 N N45 N9 N35 N8 N225 N27 N35 P P45 P9 Q Q45 Q9 R R45 R9 K Q B N R P Sample Figure 5.7: Rotational symmetry from a top-down perspective between chess pieces at a height of 92cm. 7

22 Amount of Visible board Constraining Required height to sensors area axis view area (in cm) 8 by 8 vertical axis 8 tan(2.5) = by 4 horizontal axis 8 tan(29) = by 4 vertical axis 9 tan(2.5) = 228 Table 5.2: Overview of the height required to view the playing field for, 2 and 4 sensor(s). Number of sensors Required height (in cm) = = = 29 Table 5.3: Required height to view pieces on the playing field. 5.2 Capturing the board For full visibility of the board the sensor has to be placed high, with a vertical viewing angle of 43 vertically and 58 horizontally the required height and amount of sensors can be determined using geometry[]: tan( 2 viewingangle) = 2 surface distance (5.) 2 distance = tan( 2 viewingangle) (5.2) Where viewingangle is the angle of the constraint axis, surface is the length of the viewing area, and distance is the distance from the sensor. All lengths and distances supplied in cm and the angle in degrees. The heights given in Table 5.2 are required for a full view of the board area that needs to be visible per sensor. The height required to view the playing area however requires an additional 63 cm, 63cm is the height of the king which is the tallest chess piece on the playing field. This results in the sensor placement heights described in Table 5.3. It should be noted that in the optimal positioning none of the sensors are placed directly over a square, instead they are centered over 4 squares resulting in a slightly skewed perspective of pieces placed on the playing area. Due to the symmetry of the playing field the number of squares that must be trained is vastly reduced, only a quarter of the playing field has to be learned for the model, it is however suspected that similarity ratings will go down due to side perspective on asymmetrical pieces such as the knight or king. In Figure 5.8 the sensor placement is highlighted in each of the described cases. The case where only a single sensor is used is ignored due to the fact that the required height for placement of the sensor would result in the pieces being outside the operational range of the Kinect, see

23 Figure 5.8: Position of the sensors with respect to the playing field. Sensors are shown as small rectangles in the center of a square outlined in the same color. 9

24 % & ' (! " # $ Figure 5.9: A quarter of the playing field, only a quarter of this will be used to build the reference model and test the classifier. Figure 5.: Four pawns in the top right quadrant of the playing area, the histogram was equalized to increase visibility of the pieces and the respective skew. The white spots in this image are areas that no depth could be determined by the sensor. 5.3 Four sensors, quarter board 5.3. Setup To test the effectiveness of the system in a possible real-world scenario this test focusses on the situation where the sensor is placed on a quarter of the chessboard taking care of a 4 by 4 square area. The sensor is aligned with the vertical and horizontal axis matching the board s natural segmentation, see Figure 5.9. The sensor is placed at a height of 289cm, note that this is slightly less then the required height as described in Table 5.3, which was the closest approximation of the height that could be reached in a controlled environment. This setup places the sensor centered over a 4 by 4 square of the board leaving 6 possible positions that may be required for learning, however symmetry of the board in respect to the sensor results in the need to only record a quarter of this playing field since the other three quarters are similar enough to use the same model as the trained quarter. This leaves a 2 by 2 square for training purposes, squares C, D, C2 and D2 in Figure 5.9, all of which are viewed from a skewed perspective from the camera, an example of this is shown in Figure 5.. The reference model used is once again a rotation of and it is tested against all other 2

25 references as detailed in Table 5.. The histograms used for the reference model are initially made from 3 bins. Square C2 is closest to being underneath the sensor and therefore offers the most top down view of all squares available. Square D offers the most extreme skew available in this situation Results In Figure 5.a it can be seen that the algorithm holds even under a slight skew, although there is a noticeable decline in knight recognition this can be credited to the fact that the new situation is sensitive to the orientation of the chess pieces. Mostly so with the knight which has a side profile that is drastically different from the front or back. Figure 5.b however shows that there is a severe problem detecting several pieces the moment that the skew is increased to the local extreme. In order to determine the effect of the number of bins in the model on the accuracy of the classifier, the focus lies mostly on the D square which is most susceptible to perspective changes of the piece profile. The accuracy is determined as the number of correct classifications divided by the total number of classifications. accuracy(bincount) = #correct #total (5.3) Figure 5.2 shows the accuracy for bin counts between 4 and 3 of the model, as can be seen counts of 7, 9 and 2 perform best for the current data. However a lower bin count is more robust against small shifts in the position of the pieces. In Figure 5.3 the resulting similarity scores are shown, using a bin count of 7. Compared to Figure 5.b the classifier shows greater distinction between pieces and only has significant issues with the knight on a rotation of 225. Depending on the height of the pieces and the placement of the sensor the number of bins used in the model must be adjusted Black pieces Black pieces were no longer visible to the sensor at the height of 289 cm, this is attributed to the polished black finish which most likely disrupts the projected grid used by the Kinect, see 3.2. Due to this issue black pieces are no longer taken into consideration for the evaluation of the system. 2

26 TopDown/H6 P45 P TopDown/H6 P9 P TopDown/H6 Q Q TopDown/H6 Q45 Q TopDown/H6 Q9 Q TopDown/H6 Datafile R Class R Result K Q B N R P TopDown/H6 TopDown/H6 R45 B R B TopDown/H6 TopDown/H6 R9 B45 R B TopDown/H6 B9 B TopDown/H6 K K Type evaluation TopDown/H6 K45 K TopDown/H6 K9 K TopDown/H6 K35 K TopDown/H6.875 N N TopDown/H6 N45 N TopDown/H6 N9 N TopDown/H6.625 N35 N white pieces, white model D, H289 TopDown/H6 N8 N TopDown/H6 N225 N TopDown/H6 N N TopDown/H6 N35 N TopDown/H6.25 P P TopDown/H6 P45 P TopDown/H6.25 P9 P TopDown/H6 Q Q TopDown/H6 B Q45 B45 B9 Q K K K9 K N N N9 N N8 N N27 N P P45 P9 Q Q45 Q9 R R45 R9 TopDown/H6 Q9 Q K Q B N R P Sample TopDown/H6 R R TopDown/H6 R45 R TopDown/H6 R9 R Similarity Type evaluation Similarity B B45 B9 K K45 K9 K35 N N45 N9 N35 N8 N225 N27 N35 P P45 P9 Q Q45 Q9 R R45 R9 K Q B N R P Sample (a) Results of similarity scoring for the data in square C2. (b) Results of similarity scoring for the data in square D. Figure 5.: Matching the models from a height of 289cm using a 3 bin model. 22

27 Accuracy for square D.925 Accuracy Bin Count Figure 5.2: Accuracy of the classifier using a different number of bins in the reference model. This to illustrate the effect of model size on the classifier. 23

28 TopDown/H6 N45 N TopDown/H6 N9 N TopDown/H6 N35 N TopDown/H6 N8 N TopDown/H6 N225 N TopDown/H6 N27 N TopDown/H6 N35 N TopDown/H6 P P TopDown/H6 P45 P TopDown/H6 P9 P TopDown/H6 Q Q TopDown/H6 Q45 Q TopDown/H6 Q9 Q TopDown/H6 R R TopDown/H6 R45 R TopDown/H6 R9 R white pieces, white model D, H289, bin=7 Type evaluation Similarity B B45 B9 K K45 K9 K35 N N45 N9 N35 N8 N225 N27 N35 P P45 P9 Q Q45 Q9 R R45 R9 K Q B N R P Sample Figure 5.3: Results of similarity tests with a bin count of 7 for square D. 24

29 % & ' (! " # $ ) * +, Figure 5.4: The half of the board as required for a two sensor setup. 5.4 Two Sensors, half board 5.4. Setup The final situation is the situation where only 2 sensors are used, each sensor covering half of the board. This results in a situation where each sensor covers 32 squares, once again due to symmetry the number of squares that must be learned can be reduced to 8. Figure 5.4 shows the situation from a top down perspective. In order to capture both left and right sides of the playing field the sensor must be placed at a height of 3.88 meters, for this test however the closest approximation of that height was 3.84 meters, taking into consideration that the smallest chess piece is 43 cm this means that this chess piece in on the very edge of the sensor range. However since the sensor works over a continuous range values will be retrieved even if with reduced quality in the sensor readout. To determine if the classifier still holds, even with reduced quality data from the sensor, two locations are examined, E2 and H offer the respective best and worst case scenarios of this situation, see Figure Results The first problem from this height is that there are very few bins used for the model. The value range occupied by the king, the tallest piece, in square H only occupies the range of available depth values leaving just 5 different values to describe the piece, see Figure 5.5, as opposed to the range of 5 values used for the model in Figure 5.2. This reduced range of values provides us with increased robustness agains off center placement because the sensor is incapable of determining the slight shifts in location. The downside is that the models themselves start to resemble each other more making the model more prone to misclassification. Figure 5.6 shows that the classifier performs well above expectations even with an extreme skew and low depth accuracy. The case of square E2 is skipped because it reaches an accuracy of. regardless of the number of bins used for the model. In order to explain the match a look at the model is required, in Figure 5.7 they are visualized. Due to the lesser amount of skew in the image the model is more in line with those of other top down perspectives, see Figure 5.6 and 5.2, the model created at this height closely resembles those of earlier tests but is more compressed. Because of this a two sensor setup is determined to be a feasible solution. 25

30 . BB N. N P KK Q. Q R NN P P QQ R R PP RR King model Queen model Queen Queen model model Queen model (a) Knight Depth Kingvalue model Bishop Depth Bishop value model model Bishop model Pawn Depth model value Depth value (c) Rook Bishop model Rook model Rook model Depth value King model King model King model model made from white pieces H384, bord 4x8, bin Depth (b) Bishop value Queen model Knight Depth Knight value model model Knight model Rook 986 model Depth value Pawn (d) model Pawn Knight model Pawn model Depth value Ki model made model from made white from p.25 model made made from from white white pie p D Kn D Pa D (e) Rook (f) Pawn Figure 5.5: The models for the different pieces in square H using a bin count of 9. 26