Qualitative Case-Based Reasoning for Humanoid Robot Soccer: A New Retrieval and Reuse Algorithm

Transcription

1 Qualitative Case-Based Reasoning for Humanoid Robot Soccer: A New Retrieval and Reuse Algorithm Thiago P. D. Homem 1,2, Danilo H. Perico 1, Paulo E. Santos 1, Reinaldo A. C. Bianchi 1, and Ramon L. de Mantaras 3 1 Centro Universitário FEI, São Bernardo do Campo, SP, Brazil 2 Instituto Federal de São Paulo, São Paulo, SP, Brazil 3 Artificial Intelligence Research Institute, Barcelona, Spain {thiagohomem,dperico,psantos,rbianchi}@fei.edu.br mantaras@iiia.csic.es Abstract. This paper proposes a new Case-Based Reasoning (CBR) approach, named Q-CBR, that uses a Qualitative Spatial Reasoning theory to model, retrieve and reuse cases by means of spatial relations. A qualitative distance and orientation calculus (EOPRA) is used to model cases using qualitative relations between the objects in a case. A new retrieval algorithm is proposed that uses the Conceptual Neighborhood Diagram to compute the similarity measure between a new problem and the cases in the case base. A reuse algorithm is also introduced that selects the most similar case and shares it with other agents, based on their qualitative position. The proposed approach was evaluated on simulation and on real humanoid robots. Preliminary results suggest that the proposed approach is faster than using a quantitative model and other similarity measure such as the Euclidean distance. As a result of running Q-CBR, the robots obtained a higher average number of goals than those obtained when running a metric CBR approach. Keywords: Case-Based Reasoning, Qualitative Spatial Reasoning, Humanoid Robots. 1 Introduction Traditionally, in Case-Based Reasoning (CBR) the spatial representation of a problem is given by means of a metric coordinate system, whereas the assessment of case similarity, during the retrieval step, is the main focus. As a result of that, there is a large number of distinct similarity measurement strategies based on quantitative distance functions and other metric information [5]. In some domains, however, a metric representation is not the most effective. For instance, in a humanoid robot domain, where a video camera is the main

2 2 T. P. D. Homem et al. source of information, the use of a metric coordinate system to represent object s position generates a high error rate. In this context, qualitative relations between entities can provide a more appropriate representation of the robot s environment. From the spatial distance and direction obtained by the sensor, qualitative spatial regions can be created, allowing reasoning about, and comparison of, relations between domain objects, the regions they are located and their regions of occupancy. This paper proposes a novel CBR approach using Qualitative Spatial Reasoning (QSR) to model cases and to serve as the basis of retrieval and reuse algorithms. The idea is to model the domains using EOPRA, a QSR approach that aims the representation of orientation and distance between objects with respect to the intrinsic direction of the agents [18]. Instead of representing cases using the Cartesian coordinate system, we use a qualitative orientation and a qualitative distance representation consisting of 8 qualitative orientation regions and 6 qualitative distance regions. In this context, the proposed algorithms use the concept of Conceptual Neighborhood Diagram (CND) [11, 6] and a cost function to compute the similarity measure between the problem and the case base, to retrieve the most similar case to a given situation and to reuse its solution to solve the new problem. The present work was evaluated in the Robot Soccer domain, as defined by the RoboCup Federation Humanoid League [24]. In this domain, a team of humanoid robots plays a soccer game against an opponent team. Two types of experiments were performed: the first was conducted in simulation software, in which the proposed approach was compared to the quantitative approach described in [25] and to a reactive approach; and second, experiments were executed with real robots where the present work was compared with a reactive approach. In both experiments, the number of goals scored and the retrieval time were analyzed. [25] uses the CBR approach for coordinated action selection in robot soccer domain, using the Cartesian coordinate system to represent the position of objects in the field. The present work differs from [25] since it discretizes the world into spatial representation and proposes a faster retrieval algorithm that can be used in robots with limited processing power. Finally, running the algorithms proposed in this paper, the robots performed a slightly higher average number of goals when running quantitative CBR approach. In the remainder of this work we present the CBR and QSR approaches (Section 2), the proposed Qualitative Case-Based Reasoning method (Section 3), results obtained during the retrieval and reuse steps (Section 4) and related work (Section 5). 2 Research Background This section presents the two methodologies that are used in this work, the CBR and the QSR.

3 Qualitative Case-Based Reasoning Applied to Humanoid Robot Soccer Case-Based Reasoning The essence of Case-Based Reasoning (CBR) [1] can be summarized by means of two principles of the nature: the real-world regularities (similar problems have similar solutions) and the tendency to encounter similar problems [14]. Given a new problem, CBR uses knowledge of previous situations (cases) by finding a similar past case, and reuses its solution to solve the new problem. In the robot soccer domain, a case can be defined as a triple [25]: case = (P, A, K), (1) where the problem description (P ) corresponds to the situation in which the case can be used, the solution description (A) is composed by the sequence of actions that each robot should perform to solve the problem and the case scope (K) defines the applicability boundaries of the cases. [25] proposed a retrieval method in which the similarity is evaluated along three important aspects: the similarity between the problem and the case, the cost of adapting the problem to the case and the applicability of the solution of the case. CBR has been used by several researchers in the robotic soccer domain. In addition to the work of [25], several others can be mentioned: [15] presented one of the first architectures that includes a deliberative CBR system for soccer playing agents; [13] presented high-level planning strategies, which included a CBR system. [17] presented three case-based reasoning prototypes developed for a team in the RoboCup small size league, in which CBR was used to position the goalie, select team formations and recognize game states for the team. More recently, [10] used CBR in a RoboCup soccer-playing agent playing in the Simulation League, where the agent builds a case base by observing the behavior of existing players and determining the spatial configuration of the objects the existing players pay attention to and [4] proposed a representation based on fuzzy histograms of objects and similarity metric based on the Jaccard Coefficient to compare the histograms. Finally, [2] proposed an architecture to control more complex soccer behaviors such as dribbling and goal scoring applied to humanoid multi-robot scenarios. In some papers, the approaches are analyzed in simulated environments, under optimal conditions, with an overview of the environment and without considering robot failures. The present work differs from those cited above mainly due to four reasons: in our approach (1) the agents have local vision; (2) the use of QSR approach allows an easy and fast way to retrieve and reuse cases; (3) even if the qualitative position of an object is different from the true object region, the retrieval algorithm retrieves the case with the lowest adaptation cost; and (4) the evaluation of this work was conducted in both, simulated and real environments.

4 4 T. P. D. Homem et al. 2.2 Qualitative Spatial Reasoning Qualitative Spatial Reasoning (QSR) is a subfield of knowledge representation in AI that assumes qualitative spatial relations between objects, aiming to model the human common sense understanding the space [28]. Among the several proposed formalisms in the QSR literature, for the humanoid robot soccer domain, EOPRA m best describes the positions of objects from the point of view of a robot. EOPRA m is a formalism that assigns an intrinsic orientation to the objects and refers to qualitative distance based on an elevated point from the observer [18]. A granularity parameter m allows the definition of angular zones used to represent a world discretization. Given the granularity parameter m, the soccer field is partitioned into 4m regions for each oriented object. The distance between objects is defined by means of an elevation above the 2D-plane, representing, for instance, the viewpoint of an human observer and the way she visually perceives the world. So, the distance between objects is obtained projecting their elevation onto the 2D-plane [7]. The granularity parameter n for the distance model also allows a discretization of the world according to the needs of the application domain, creating 2n sectors. For each QSR formalism, a specific Conceptual Neighborhood Diagram (CND) can be defined as a graph that represents a jointly exhaustive and pairwise disjoint set of qualitative relations, where the nodes correspond to a relation between two spatial entities and the edges correspond to a pair of conceptual neighbors (i.e. there is no other relation from the set that represents the transition from the pair). [23] have used CND as a tool to compare and measure the distance between sets of spatial regions and create a similarity matrix. CND can also be used in qualitative simulations [6]. In this paper, CND is used as a tool to measure the distance between cases and to retrieve the most similar case. 3 Problem Formulation This section presents the Qualitative Case-Based Reasoning (Q-CBR) method, the qualitative spatial modeling for the cases, the CND of EOPRA m and the description of the use of CND as a tool for similarity measuring, creating a new retrieval algorithm for CBR. 3.1 Qualitative approach to represent direction and distance This work uses EOPRA m to represent the relation between any two objects as a tuple of orientation and distance. Based on the work of [18] and [7], we have considered the viewpoint orientation as being the front of the agent and the granularity parameter m = 6, creating 24 direction sectors. These direction sectors are grouped into 8 regions: left, right, front, back, left-front, right-front, left-back and right-back. Figure 1a shows the direction sectors and regions created. For each of the front, left, back and right regions is obtained an angular

5 Qualitative Case-Based Reasoning Applied to Humanoid Robot Soccer 5 Fig. 1: (a) Qualitative direction representation. (b) Qualitative distance representation. region of 60 o and to the remainder regions, left-front, right-front, left-back and right-back, angular regions of 30 o. We have considered the left, front, right and back regions as being more important orientation regions than others, so they have an angular region of 60 o. Regarding the elevated point and distance relations, a granularity parameter of n = 6 was assumed, creating 12 distance sectors. These distance sectors are then grouped into 6 categories: at, very close, close, far, very far and farthest. Figure 1b shows the distance regions created. Based on [18] and in the agent s height (0.55 meters), the regions were defined as: at refers to an object placed at less than 0.33 meters, very close is to an object placed between 0.33 and 0.66 meters, close is to an object placed between 0.66 and 1.00 meter, far is to an object placed between 1.00 and 1.50 meters, very far is to an object placed between 1.50 and 3.00 meters, and farthest refers to an object at more than 3.00 meters. Figure 2a presents the qualitative discretization created, in which the orientation and distance has granularity parameter m = 6, named EOPRA 6. At the center of EOPRA 6, a region labeled equal corresponds to the agent s position and the position of any object to the agent. Figure 2b presents the CND of EOPRA 6. The nodes describe all qualitative relations and the edges describe its transformation to another relation. Now, similarly to the work of [23], it is possible to define a distance function Dmin φ (X 1, X 2 ) that takes two spatial relations X 1 and X 2 and maps them to the minimum CND (node to node) distance between them. This distance can be computed using any algorithm to find the shortest path between nodes in a graph, such as Dijkstra s algorithm [3]. Using this distance function, a distance matrix for CND can be created with the minimal CND node-node path distance between the 49 relations, allowing a quick retrieval during distance calculation 4. 4 The distance matrix for EOPRA 6 is available at the URL

6 6 T. P. D. Homem et al. Fig. 2: (a) Qualitative representation for distance and direction. (b) CND of the proposed EOPRA 6 representation. 3.2 Qualitative case representation Inspired by the work of [25], a case (C ) is defined as: the problem description (P) and the solution description (A): C = (P, A). (2) The problem description (P ) corresponds to the qualitative spatial relation descriptions between an agent and the objects in the environment, given by the qualitative direction and distance to each object, from the agent viewpoint. P is given by: P = {R 1 : [O 1, O 2,...O u ],..., R v : [O 1, O 2,...O u ]}, (3) where v is the number of agents in the problem, u is the number of objects that each agent can perceive, R i is the number of the agent and O 1, O 2,..., O u are the qualitative relations between the object and the current agent (each one being an orientation and distance tuple). By objects, we mean the ball and other robots that can be seen by the agent. As in [25], the solution description (A) describes a sequence of actions each agent must perform to solve the problem, as shown in expression 4: A = {R 1 : [a 11, a 12,..., a 1p1 ],..., R v : [a v1, a v2,..., a vpv ]}. (4) Differently to [25], the use of the case scope (K) is not necessary to the qualitative representation of cases, therefore, the qualitative spatial position of objects in the environment is a region and not a point on the coordinate plane. 3.3 Qualitative case retrieval In general, the retrieval step consists of measuring the similarities between the new problem and the solved problems stored in the case base. The present work uses the distance between objects in the CND to compute the similarity between

7 Qualitative Case-Based Reasoning Applied to Humanoid Robot Soccer 7 the new problem and the cases in the case base. This can be done using the distance function used to compute the distance matrix presented in Section 3.1. The qualitative distance function is defined as: v u Dist Q (p, c) = Dmin φ (R c i, R p i ) + Dmin φ (O c j, O p j ), (5) i=1 where v is the number of robots that take part in the case solution, u is the number of objects that each agent can perceive, R i c the qualitative position of each robot i in the case and R i p its qualitative position in the problem, O j c the qualitative position of each object j in the case and O j p its qualitative position in the problem. The qualitative similarity function is defined as: Sim Q (p, c) = CND MaxDist (v + u) Dist Q (p, c), (6) CND MaxDist (v + u) where v and u are as defined in the qualitative distance function and CND MaxDist is the maximum distance between two objects in the CND. The result is normalized, so the similarity is bounded between 0 and 1. The qualitative adaptation cost function is defined as: v Cost Q (p, c) = Dmin φ (R c i, R p i ), (7) i=1 where v is the number of robots that take part in the case solution and R i c the qualitative position of each robot i in the case and R i p its qualitative position in the problem. The adaptation cost function includes only robots that are of the same team as the agent, meaning that their position can be controlled (i.e., adapted). The adaptation cost is the cost to move the robots of the team to the position that is described in the most similar candidate case, and it reflects how much would cost to adapt the position of the robots in the world to the positions in the most similar candidate case. Algorithm 1 presents the proposed retrieval method based on CND distance measure and adaptation cost. In this algorithm, there are two lists: sim candidates which contains cases that are above a minimum similarity value (threshold); and the list adapt candidates that is used to compute the adaptation cost of the candidate cases, ordered by their cost. Lines 2-11 search for candidate cases in the entire case base. Line 3 measures the qualitative similarity from problem to case using Equation 6. In lines 4-5, if a case that is equal to the problem is found, the function returns it and ends the search. If no case is found within the similarity range allowed, a pre-defined, reactive case is returned (lines 12-13). Lines compute the cost of adaptation of each case that was found in the previous steps, sort them by the adaptation cost, and return the one with the lowest adaptation cost (sim value is the second sort criteria). The reactive behavior returns when no similar case is retrieved (lines 12-13); this consists of a naïve behavior, in which the robot searches for the ball and walks toward it, aligns itself with respect to the opposing goal and kicks forward. j=1

8 8 T. P. D. Homem et al. Algorithm 1 Retrieval step using CND similarity measure. 1: function Retrieve(Problem p, Case base CB) 2: for each case c CB do 3: sim value Sim Q(p, c) 4: if sim value = 1 then 5: return c 6: else 7: if sim value > T hreshold then 8: insert(sim value, c, sim candidates) 9: end if 10: end if 11: end for 12: if empty(sim candidates) then 13: return reactive case 14: end if 15: for each case c sim candidates do 16: adapt value Cost Q(p, c) 17: insert(adapt value, c, adapt candidates) 18: end for 19: sort(adapt value, adapt candidates) 20: return first(c, adapt candidates) 21: end function 3.4 Qualitative case reuse The reuse step consists of adapting the position of the robots in the problem to the qualitative position of the retrieved case. Basically, this step contains three agents: the coordinator robot (R coord ), which coordinates the retrieval and reuse steps, the executor robot (R exe ), a robot that is part of the solution, and a retrieved robot (R ret ), a virtual robot which represents the R exe s position of the retrieved case. The reuse step is focus on calculate the how R exe can reach to R ret s position and the actions it must perform to reach for the position. So, the Composition Algorithm (CA) of [21] was used to calculate the qualitative orientation and distance from R exe to R ret. The CA uses an extension of EOPRA m where distance inference is made by a quantitative triangulation using the law of cosines and direction is inferred by the traditional OPRA m [19] restricted by quantitative data. Algorithm 2 presents the proposed reuse method that uses OPRA m composition restricted by quantitative triangulation [21]. As the retrieved case contains the qualitative position of the coordinator robot s point of view, it needs to be converted to the executor robot s point of view, that has its own qualitative relations about the world. The algorithm receives the problem and the retrieved case and, for each robot that is part of the solution, an adapted position is generated based on the executor robot s point of view (line 3). Line 4 shares with the executor robot the adapted positions it needs to act to the estimated position and line 5 shares the actions it must perform to solve the problem.

9 Qualitative Case-Based Reasoning Applied to Humanoid Robot Soccer 9 Algorithm 2 Reuse step using Composition Algorithm. 1: function Reuse(Problem p, Case c) 2: for each robot r executors robot do 3: adapt pos Composition Algorithm(p, c, coord, r) 4: send positions(adapt pos, r) 5: send actions case(c.a) 6: end for 7: end function Fig. 3: Example of Reuse step using Composition Algorithm In order to exemplify the Reuse step using CA, Figure 3 presents the coordinator robot s (R coord ) point of view about the executor robot s (R exe ) qualitative position and the robot s position on the retrieved case (R ret ), and the executor robot s point of view about the coordinator robot s qualitative position. R coord can easily obtain the angle β, so it can calculate the angle α using the law of cosines. After obtaining α, this angle is discretized according OPRA 6 definitions, representing the R exe s qualitative orientation to the R ret position. The R exe s qualitative distance is calculated by Pythagorean theorem and the distance is discretized according EOPRA 6. In Figure 3: (1) the R coord searches for the objects position on the environment and finds the R exe s position in left,farthest; (2) it retrieves a case and selects the most similar case where the robot s position in the case is front,very far (R ret ); (3) by running the Composition Algorithm, it calculates the adapted position to the R exe s point of view (right-front,farthest) and shares to it; (4) R exe executes the movements to reach to R ret s position and (5) performs the actions to solve the problem. 4 Experiments and Results This section presents the experiments and results obtained with the algorithms introduced in this work applied to the humanoid robot soccer environment. Two types of experiments are performed: (1) in a simulator, where Q-CBR approach, the metric approach of [25] and a reactive agent were compared; (2) in real humanoids robots, where a comparison between Q-CBR and the reactive agent was conducted. The experiments in this section aim to analyze which of the approaches resulted in more goals scored and fewer errors, and to compare the retrieval time of cases between quantitative and qualitative approaches. The following subsections present the experiments and the results obtained.

10 10 T. P. D. Homem et al. Fig. 4: (a) Simulated scenario #1. (b) Simulated scenario # Simulation Experiments Simulation experiments were conducted using a software developed with the purpose of enabling the reproduction of experiments and performance comparison of different algorithms in the literature: the RoboFEI Humanoid Soccer Simulator. This simulator uses the Cross architecture described in [22], which implements low-level processes, such as vision, control and communication processes, allowing users to develop and test high-level AI algorithms as collective strategies or decision-making processes in simulation. The simulator also facilitates the code to be transferred to real robots without the need of many modifications. The RoboFEI Humanoid Soccer Simulator is an open-source simulator, written in Python, which allows the integration with other programming languages like C and C++. The simulator environment is a football field that follows the rules of RoboCup Humanoid Kidsize [24], with two robot teams allowing the user to develop different strategies for each robot. The simulated experiments were performed in an Intel NUC i5 with 8GB SDRAM running Ubuntu LTS. For reproducibility reasons, the simulator used in this work, along with the source code of the proposal, are available at the URL ~rbianchi/software.html. Two scenarios were created for the experiments, as shown in Figure 4. In the first one (Figure 4a) the ball and the robot are positioned in the center of the field and a teammate is positioned to the left and in the middle of the field. There are also three opponents positioned as defenders and a goalkeeper. In the second scenario (Figure 4b), the ball, the robot and one teammate are positioned in the attacking field and four opponent robots are positioned similarly to scenario #1. In both scenarios we used a centralized case base, in which the closest robot positioned to the ball assumes the position of coordinator, being responsible for the retrieval process (in qualitative and quantitative approaches) and for the coordination of collective actions in the reuse process. The coordinator robot transmits wirelessly the adapted positions and actions that the robots must perform, which are received and executed by the other robots, called executors. In order to perform the experiments, two case bases were created and populated: (1) a quantitative case base: with 20 real cases and 180 random cases,

11 Qualitative Case-Based Reasoning Applied to Humanoid Robot Soccer 11 with random positions and three actions for each robot, and (2) a qualitative case base: with the same 200 cases represented as qualitative relations. The 20 hand-coded cases represent specific positions of the robots and the actions each robot must performs to solve a problem, such as a setplay. In the reactive approach, only reactive actions were implemented, in which the robot looks for the ball, walks toward it, aligns and kicks the ball, with completely uncoordinated behavior. Although the world discretization presented in Section 3.1 defines 8 qualitative regions of direction, during the experiments, only 7 qualitative regions were used due to the RoboCup rules that define the mechanism to pan the camera limited to ±135, discarding the region named as back. For comparison purposes, 40 trials of 10 minutes were performed for each scenario and for each algorithm tested. In each trial, we considered the number of goals obtained, the number of near misses and the number of errors (for example, when the robots cannot find the ball or the sequence of coordinated actions do not result in a goal). Table 1 shows the results obtained for each of the algorithms tested. Q-CBR obtained a slightly higher average number of goals when compared to the metric algorithm. Both Q-CBR and the metric algorithm outperform the reactive agent in the scenarios considered. Student s t-test[20] was applied in each scenario and the results indicate that the Q-CBR is statistically better in most cases (with a certainty of at least 99%). Another advantage of using Q-CBR is the case retrieval time. The results presented in Table 2 show that Q-CBR is about 3 times faster than the metric algorithm, and it allows the implementation in humanoid robots with limited processing power and hardware. The improvement in the retrieval time is due to the strategy for the qualitative similarity measurement, as shown in algorithm 1. Student s t-test was also used in order to compare the computational performance of Q-CBR to the metric algorithm and the results (Table 2) indicate that the Q-CBR is statistically better than the quantitative CBR with a certainty of at least 99%. 4.2 Experiments with real robots The use of qualitative relations to represent spatial positions for real humanoid robots is an advantage of Q-CBR. In this domain, the robots do not know the Table 1: Results of simulation experiments (mean and standard deviation for 40 trials of 10 minutes). Scn. Method Goals T-value Near Misses T-value Errors T-value CBR 2.58 ± ± ± (90%) 7.72 (99%) 5.39 (99%) #1 Q-CBR 2.98 ± ± ± 0.99 Reactive 0.33 ± ± ± CBR 2.55 ± ± ± (95%) 2.43 (99%) 4.92 (99%) #2 Q-CBR 3.10 ± ± ± 0.84 Reactive 0.48 ± ± ±

12 12 T. P. D. Homem et al. Table 2: Performance of the CBR and Q-CBR retrieval step (Time in seconds, averaged over 40 trials; absolute t-value and confidence interval (in %)). Scenario Method Retrieval (seconds) T-value #1 CBR ± Q-CBR ± (99%) #2 CBR ± Q-CBR ± (99%) global position of the agents in the field because, unlike other domains, the robot uses the camera as its primary recognition sensor. Thus, the qualitative spatial representation of the robots and the ball position becomes an easy way to model, retrieve and reuse cases in the case base. These experiments were conducted with two humanoid robots based on the Darwin-OP robot, adapted to use a computer with the same configuration of the simulation experiments. The scenario was similar to the scenario #2 in simulation experiments, with the same case base. Using the same implementation of the qualitative and reactive approaches in the simulator, the implementation on real robots did not require many changes, so only the Vision and the Control modules of the Cross Architecture were changed. Thus, the robots were able to recognize the ball and other robots, communicate with each other and perform basic tasks like walking, turning, kicking and passing. The experiments consist of 5 trials of 10 minutes and, as in the previous experiments, the average number of goals, the number of near misses and the number of errors were considered. Table 3 presents the results of Q-CBR and the Reactive algorithm. Experiments were not conducted with the metric CBR algorithm due to the fact that, in contrast to the Simulator, coordinates of the robots and the ball in the field are not given in the real-robot scenario. The average retrieval time is similar to the simulation (about seconds), although the number of goals scored could be higher with the improvement of some aspects of the robot, such as the control of walk, kick or pass. Student s t-test was used again in order to compare the performance of the proposed approach with the reactive algorithm. In this experiment, the Q-CBR is statistically better for scored goals than the reactive approach with a certainty of at least 95%. Table 3: Results with real robots experiments (mean and standard deviation for 5 trials of 10 minutes). Method Goals Near Misses Errors Q-CBR 1.20 ± ± ± 1.16 Reactive 0.40 ± ± ± 1.02

13 Qualitative Case-Based Reasoning Applied to Humanoid Robot Soccer 13 5 Related Work Several CBR work can be found in the literature using cases with qualitative representation but with no relation to QSR approaches. For instance, [8] present an algorithm to integrate spatial relations into CBR, extracting the similarity coefficient of cases and problem and matching each other with respect to some characteristic. The work of [16] proposes a CBR algorithm based on qualitative causality. The work reported in this paper uses QSR approach to represent the objects position and it retrieves the most similar case based on a CND. So, the neighborhood diagram allows us to define the distance between relations and to calculate an adapted position to the agent. On the other hand, the work of [12] uses temporal reasoning and CBR, where the cases are represented as temporal graphs and the retrieval step is performed matching the graphs and creating a similarity degree. [9] propose an approach for adaptation of spatial and temporal cases during the reuse step of CBR, where the similarity between two scenarios is measured based on the distance between the considered relations. It differs from our retrieval proposal since we compare each qualitative position of the objects in cases with the objects in the problem, retrieving the cases that have the minimal cost of adaptation among the cases that have the most similar CND to the CND of the problem. The work of [30, 29] applied the Star Calculus to represent the qualitative direction between entities on the RoboCup Soccer Keepaway [27]. In another environment, [26] applied QSR to games, where the objects position were modeled as qualitative spatial relations. The results of these papers show that the use of QSR is an interesting way to generalize the objects position representation. Our work uses EOPRA and compares its retrieval time to a metric algorithm. We also perform experiments on real robots, with limited view of the environment. 6 Conclusion This work showed that by modeling cases in a CBR system as qualitative spatial relations and using the CND similarity and cost functions as similarity measure, we have obtained a faster and easier way to retrieve a case, with a better performance than using a more traditional, metric, model. In some domains, the use of qualitative representation is more appropriate than using quantitative values. The humanoid robot soccer is one of these domains, as the robots cannot obtain a precise position of the objects in the field. Aiming to analyze the proposed method, we performed the initial experiments in a simulated environment with a small case base, using two distinct scenarios. We also ran our proposal in real humanoid robots. The results show that the proposed method increases the number of scored goals and decreases the average time spent to retrieve a case. In all experiments, the algorithm introduced in this paper (Q-CBR) has been about 3 times faster than the metric algorithm tested, allowing to execute the Q-CBR in robots with a limited processing power and limited hardware.

14 14 T. P. D. Homem et al. As future work, we propose to implement the complete Q-CBR cycle and analyze the results of the revision and retention processes. We also propose to implement Q-CBR as a multi-agent system, where each robot has its own case base and cooperates with the other team members to define which case would better solve the problem. The planning process of our Cross architecture will also be extended with motion planning, allowing the robots to move to an adapted position in an optimal trajectory, for instance. 7 Acknowledgements Thiago P. D. Homem acknowledges support from CAPES and PRP/IFSP. Danilo H. Perico acknowledges support from CAPES. Paulo E. Santos acknowledges support from FAPESP (2012/ ). Ramon L. de Mantaras acknowledges support from Generalitat de Catalunya Research Grant 2009 SGR 1434 and CSIC Project E022. References 1. A. Aamodt and E. Plaza. Case-based reasoning: Foundational issues, methodological variations, and system approaches. AI communications, 7:39 59, M. M. Altaf, B. M. Elbagoury, F. Alraddady, and M. Roushdy. Extended casebased behavior control for multi-humanoid robots. International Journal of Humanoid Robotics, T. H. Cormen, C. E. Leiserson, R. L. Rivest, and C. Stein. Introduction to Algorithms. MIT Press, 2nd edition, Alan Davoust, Michael W. Floyd, and Babak Esfandiari. Use of Fuzzy Histograms to Model the Spatial Distribution of Objects in Case-Based Reasoning, pages Springer, R. L. de Mántaras, D. McSherry, D. Bridge, B. Smyth, S. Craw, B. Faltings, M. L. Maher, M. Cox, K. Forbus, M. Keane, A. Aamodt, and I. Watson. Retrieval, reuse, revision, and retention in CBR. In The Knowledge Engineering Review, volume 20, pages N. V. De Weghe and P. De Maeyer. Conceptual Neighbourhood Diagrams for Representing Moving Objects. LNCS, 3770: , Springer, C. H. Dorr, L. Jan Latecki, and R. Moratz. Shape Similarity Based on the Qualitative Spatial Reasoning Calculus eopram. 12th International Conference, COSIT 2015, Proceedings, pages Springer, NM, USA, Y. Du, F. Liang, and Y. Sun. Integrating spatial relations into case-based reasoning to solve geographic problems. Knowledge-Based Systems, 33: , V. Dufour-Lussier, F. Le Ber, J. Lieber, and L. Martin. Adapting Spatial and Temporal Cases. 20th ICCBR 2012, Proceedings, pages Springer, M. W. Floyd, B. Esfandiari, and K. Lam. A case-based reasoning approach to imitating robocup players. Proceedings of the Twenty-First FLAIRS, 2008, Florida, USA, pages AAAI Press, C. Freksa. Conceptual neighborhood and its role in temporal and spatial reasoning. Decision Support Systems and Qualitative Reasoning, pages , 1991.

15 Qualitative Case-Based Reasoning Applied to Humanoid Robot Soccer M. D. Jære, A. Aamodt, and P. Skalle. Advances in Case-Based Reasoning: 6th ECCBR 2002, Proceedings, pages Springer, A. Karol, B. Nebel, C. Stanton, and M. Williams. Case based game play in the robocup four-legged league part i the theoretical model. RoboCup, volume 3020 of LNCS, pages Springer, J. Kolodner. Case-based Reasoning. Morgan Kaufmann Publishers Inc., San Francisco, CA, USA, Y. Lin, A. Liu, and K. Chen. A hybrid architecture of case-based reasoning and fuzzy behavioral control applied to robot soccer. In Workshop on Artificial Intelligence (ICS2002), Hualien, Taiwan, National Dong Hwa University. 16. Z. Liu, L. Fu, and Y. Zhou. Case-Based Reasoning Algorithm Based on Qualitative Causality. In 7th Int. Joint Conference on CSO, pages IEEE, jul C. Marling, M. Tomko, M. Gillen, D. Alex, and D. Chelberg. Case-based reasoning for planning and world modeling in the robocup small sized league. In Workshop on Issues in Designing Physical Agents for Dynamic Real-Time Environments: World Modeling, Planning, Learning, and Communicating (IJCAI), R. Moratz and J. O. Wallgrün. Spatial reasoning with augmented points: Extending cardinal directions with local distances. Journal of Spatial Information Science, 5(5):1 30, dec T. Mossakowski and R. Moratz. Qualitative reasoning about relative direction of oriented points. Artif. Intell., :34 45, April U. Nehmzow. Scientific Methods in Mobile Robotics: Quantitative Analysis of Agent Behaviour. Springer-Verlag London, D. H. Perico, R. A. C. Bianchi, P. E. Santos, and R. L. de Mántaras. Collaborative communication of qualitative spatial perceptions for multi-robot systems. In Proc. 29th International Workshop on Qualitative Reasoning (IJCAI), pages 77 84, New York, NY, USA, D. H. Perico, I. J. Silva, T. P. D. Homem, R. C. Destro, and R. A. C. Bianchi. Hardware and software aspects of the design and assembly of a new humanoid robot for robocup soccer. In 2014 Joint Conference on Robotics: SBR-LARS, pages IEEE, October D. A. Randell and M. Witkowski. Tracking Regions using Conceptual Neighbourhoods. Proceedings of the Workshop on Spatial and Temporal Reasoning ECAI , pages 63 71, RoboCup robocup soccer humanoid league rules and setup R. Ros, J. L. Arcos, R. L. de Mantaras, and M. Veloso. A case-based approach for coordinated action selection in robot soccer. Artificial Intelligence, 173(9-10): , jun T. Southey and J. J. Little. Learning Qualitative Spatial Relations for Object Classification. In IROS 2007 Workshop: From Sensors to Human Spatial Concepts, P. Stone, R. S. Sutton, and G. Kuhlmann. Reinforcement Learning for RoboCup Soccer Keepaway. Adaptive Behavior, 13(3): , D. Wolter and J. Wallgrün. Qualitative spatial reasoning for applications: New challenges and the SparQ toolbox. Qualitative spatio-temporal representation and reasoning: Trends and future directions, pages , J. Young and N. Hawes. Predicting Situated Behaviour Using Sequences of Abstract Spatial Relations. AAAI 2013 Proceedings, Fall Symposium Series, J. Young and N. Hawes. Learning by Observation Using Qualitative Spatial Relations. In Proceedings of the 2015 AAMAS, pages , Richland, SC, 2015.