Reliable Classification of Partially Occluded Coins e-mail: L.J.P. van der Maaten P.J. Boon MICC, Universiteit Maastricht P.O. Box 616, 6200 MD Maastricht, The Netherlands telephone: (+31)43-3883901 fax: (+31)43-3884897 {l.vandermaaten,p.boon}@micc.unimaas.nl Abstract 1 Introduction Over the last years, a number of successful systems for reliable classification of heterogeneous coin collections have been developed [3, 7, 8, 9, 10]. State-of-the-art systems for coin classification recognize coins by means of measurements in digital photographs of these coins, which gives these systems a large advantage over traditional coin classification systems based on thickness and weight measurements. Image-based coin classification systems can be subdivided into two types, viz., systems based on invariant features [3, 9] and systems based on template matching [7, 8, 10]. The latter type of systems has proven to be the most successful one in, e.g., the 2006 MUSCLE CIS benchmark competition. In contrast to systems for the classification of modern coins, the development of systems for the classification of historical coins has gained virtually no attention, possible due to the lack of publicly available digitized historical coin collections. Systems for the automatic classification of historical coins could be of great interest to cultural heritage institutions such as the Dutch Money and Bank Museum, because they can make numismatic knowledge available to a large public. In [10], some promising results are reported on a dataset of Merovingian coins, but these results are limited in scope. In [5], a project that aims at the automatic classification of Roman coins is discussed, but no experimental results are reported. In order to trigger the development of systems for classification of historical coins, the 2007 MUSCLE CIS benchmark competition aims at the classification of modern coins under the presence of occlusions. This paper describes our submission to the benchmark competition, which is based on the system described in [10]. The outline of the remainder of this paper is as follows. In section 2, the MUSCLE CIS benchmark competition 2007 is described in more detail. Section 3 provides a detailed description of 1
our system. In section 4, we present our experiments on the M USCLE C IS benchmark dataset. The results of our experiments are discussed in more detail in section 5. Section 6 concludes the paper. 2 M USCLE C IS benchmark competition 2007 The M USCLE C IS benchmark competition 2006 is the predecessor of the 2007 competition in which two systems for coin classification participated [8, 9]. The setup of the 2007 competition is similar to the 2006 competition, however, the test coins in the 2007 competition may be partially occluded. Four examples of partially occluded coins are shown in Figure 1. The idea behind the distortions is to simulate historical coins, in which generally only partial stamp information is available. The aim of the competition is to classify 5,000 coins (i.e., 10,000 coin images) within 12 hours Figure 1: Examples of partially occluded coins. on a normal 3GHz PC with 1GB RAM. As a result, only computational efficient techniques can be applied in the coin classification system. Approximately 3% of the coins in the testset was not in the trainingset, and needs to be classified as unknown. At least 70% of the coin classifications needs to be correct in order to participate in the competition. The performance of the system is measured by means of an assessment scheme that outputs a number of points. For each correct classification, the system receives 1 point, whereas incorrect classifications lead to a score update of -100 points. Classifications as unknown receive no points (unless the coin is truly an unknown coin). For every coin class of which at least one coin was classified correctly, the system receives an additional 25 points (in order to prevent the submission of systems that are trained only on a subset of the coin classes in the trainingdata). The assessment scheme clearly indicates the importance of reliable classifications. 3 The system The workflow of our system for classification of coins consists of three main stages: (1) a segmentation stage, (2) a feature extraction stage, and (3) a classification stage. In the segmentation stage, the coin is extracted from its background in the coin photograph. In addition, occluded parts of the coins may be recognized in the segmentation stage. In the feature extraction stage,
informative features are extracted from the segmented coin image. In our system, the features are formed by polar gradient orientations. In the classification stage, a label is assigned to unlabeled coins by means of a nearest neighbor approach. The three stages are described in more detail in subsection 3.1 to subsection 3.3. 3.1 Segmentation In our system, segmentation is performed by means of a two-stage approach. In the first stage, we attempt to segment coins by means of an edge-detection with additional morphological operations. In this stage, we assume the upperleft pixel of the coin photograph is a background pixel. Subsequently, the segmentation is verified by checking whether the segmented part is (nearly) circular and whether its area is sufficiently large. In case the segmentation has failed in the first stage, the second segmentation stage is executed. In the second stage, segmentation is performed by means of a Hough circle detector [4] that is implemented in a multi-scale manner. More specifically, we iteratively perform the circle detection using six radiuses that lie around the radius of the maximum Hough accumulator in the previous iteration. The iterations are performed until the finest level of detail (i.e., 1 pixel radius differences between the six radiuses) is reached. The segmentation allows for extracting the coin from its background in the coin photograph, but it does not identify occluded parts of the coins. The identification of occluded parts is of interest, because it may help to improve the similarity measure between coins that is discussed later. Because the conveyor belt has a regular structure, occluded regions in the coins may be identified by means of a texture classifier. In this respect, we performed experiments with texton-based texture classifiers [6, 11]. Texton-based texture classifiers perform texture classification based on a codebook of so-called textons. Textons are small patches of texture that are represented by means of, e.g., a collection of filter responses or a simple concatenation of pixel values. The texton codebook is constructed by performing vector quantization (e.g., using k-means clustering or Kohonen maps) on a set of randomly selected textons. We performed experiments with texton-based texture classifiers using a codebook of 20 pixel-based textons of size 13 13. In these experiments, we experienced a strong performance of texton-based texture classifiers for the detection of occlusions (both with respect to computational efficiency and true positive rate). The occlusion detection is illustrated in Figure 2. Figure 2: Occlusion detection by a texton-based texture classifier.
3.2 Feature extraction Results in previous studies on coin classification show that template matching approaches to coin classification generally outperform coin classification systems based on rotation-invariant features [3, 7, 8, 10]. Hence, we opt for a template matching approach to coin classification. Since the main visual information in coins is contained in its stamp, it is natural to perform the template matching based on gradient images. In previous studies, both classification based on gradient orientations and gradient magnitudes is already investigated [7, 8]. These studies give rise to the observation that gradient orientations contain valuable structured information in regions with very low gradient magnitudes, which is illustrated in Figure 3. Gradient orientations have also been successfully applied in, e.g., image registration [1] and face localization [2]. Hence, we opt for a template matching approach based on gradient orientations. In our system, feature extraction is performed by means of a two-stage approach. First, the coin Figure 3: Illustration of gradient magnitude and orientation. image is convolved with a small Gaussian kernel, and it is converted into a polar representation. The use of a polar coin representation facilitates the computation of rotation-invariant gradient orientations. Second, the polar coin image is convolved with a small Gaussian kernel, and the gradient orientations in the resulting image are computed. The gradient orientation of an image I with coordinates (x, y) is given by ( ) δi/δy Θ = arctan (1) δi/δx The reader should note that these gradient orientations correspond to tangential and slope orientations in the original Cartesian space. As a result, polar gradient orientations are invariant to rotations of the coin. In addition, the use of a polar coin representation allows for efficient computation of our similarity measure.
3.3 Classification The classification of coin images starts with a preselection based on area and thickness. If the measured radius of a coin differs more than 7 pixels from the other coins in a coin class, or its thickness differs more than 1.25 mm, the coin class is rejected. In addition, edge-based statistical features could be used as preselection features [10]. Coin classes that are not rejected are gathered in a preselection list. The classification of coin images is performed by means of a nearest-neighbor search through the preselection list, in which a similarity measure s ij between the coins i and j is employed that is given by s ij = max γ ɛ (Θ α i (x, y), Θ j (x, y)) dydx (2) α In the equation, Θ α i represents the polar gradient orientation image Θ i that is circularly shifted corresponding to a rotation of the coin by α degrees. The decision function γ ɛ (, ) determines whether two gradient orientations are similar or not. The value of γ ɛ (, ) is 1 when the absolute difference between two gradient orientations is smaller than ɛ, and 0 when the absolute difference between both gradient orientations is larger than ɛ. If both gradient orientations are undefined, the decision function γ ɛ (, ) returns 1. The function γ ɛ (, ) returns ɛ when one of π the gradient orientations is undefined, or when the location is marked by the occlusion detector. The decision function is employed to make the system robust to small variations in the gradient orientations. The maximization over the coin orientation α is necessary in order to construct a rotation invariant similarity measure. Because a coin has two sides, two classifications have to be performed, and these classifications need to be combined. We opt for a simple combination strategy in which a classification is solely accepted when both classifications correspond. In case the two classifications do not correspond, the coin is classified as unknown. This classifier combination strategy ensures reliability of the classifications. In order to increase the reliability of the classifications, we also compare the orientations α 1 and α 2 we found in the computation of the similarity measure. Due to the digitization process of the coins, these orientations should roughly correspond. We classify a coin as unknown, whenever the difference between α 1 and α 2 is larger than 70 degrees. 4 Experiments In order to evaluate the performance of our coin classification system, we performed experiments in which we measure the generalization errors of our coin classifiers on the coin dataset described in section 2. In subsection 4.1, we describe the setup of these experiments. Subsection 4.2 presents the results of our experiments on the coin dataset. 4.1 Experimental setup We evaluated the performance of our system on two datasets: (1) a normal coin testset from the 2006 competition, and (2) the occluded coin testset from the 2007 competition. Both datasets consist of 5,000 coins (i.e., they contain 10,000 coin images). In both experiments, we trained
Settings Results Dataset Occl. detection Correct Incorrect Unknown Normal No 88.86% 0.38% 10.76% Occluded No 75.63% 0.14% 24.23% Occluded Yes 75.85% 0.20% 23.95% Table 1: Performance of our approach on modern coin datasets. our system on the set of coin prototypes. In the experiments, we measure the percentage of correct classifications, the percentage of incorrect classifications, and the total computation time that is consumed for the processing of the complete dataset. 4.2 Results Table 1 presents the results of the experiments with our system on both the normal and the occluded modern coin datasets. The table shows the percentage of correctly classified coins, the percentage of incorrectly classified coins, and the percentage of coins that was classified as unknown. For the experiments on the occluded coin dataset, we show the performance of our system with and without the use of occlusion detection (see subsection 3.1). From the results presented in the table, we make two observations. First, the results reveal that the presence of occlusions in the coins degrades the performance of our coin classification system by approximately 13%. The set of coins that was not classified by the system mainly consists of coins with very low contrast and of coins with large occluded regions. Second, the results in Table 1 show that the detection of occlusions does not improve the classification performance of the system. Eliminating occluded image regions from the similarity measure even seems to slightly increase the number of incorrect classifications. 5 Discussion From the results presented in the previous section we made two main observations: (1) the presence of occlusions in coins degrades the performance of coin classification systems and (2) detection of these occlusions using texture classifiers does not improve the performance on coin datasets in which coins are partially occluded. The results in section 4 show that the presence of occlusions clearly reduces the performance of coin classification systems. The explanation for the degradation in performance caused by the presence of occlusions is twofold: (1) the amount of stamp information is reduced and (2) the occlusion borders contain very structured gradient orientations. In the occluded image regions, the number of corresponding gradient orientations with an training image is presumably nearly random. This increases the influence of noise in the remainder of the coin on the similarity measure. At the borders of the occlusions, the gradient orientations are very structured and might correspond to patterns in the stamps of other coins, leading to misclassifications. The results in Table 1 show that occlusion detection by means of a texton-based texture clas-
sifier does not improve the generalization performance of our system on a dataset of partially occluded coins. Most likely, this is due to the regularity of the texture in the occluded parts of the coin. The conveyor belt texture roughly contains an equal amount of all orientations. As a result, an occlusion causes a nearly constant term to be added to our similarity measure, of which the effect is largely mitigated by the normalization in the similarity measure. On the other hand, false positives in the texture classification may reduce the performance of the coin classification system by removing possibly important information from the similarity measure. Although our results on the dataset of partially occluded coins are promising, the reader should note that the performance of our coin classification system will be worse on historical coin collections, due to the presence of additional distortions in historical coins. For instance, in historical coins the coin stamp is often not in the center of the coin. In addition, historical coins are often highly degraded by being buried in the soil. These distortions are illustrated in Figure 4 1. (a) Coin with stamp that is out-of-center. (b) Coin that is heavily degraded. Figure 4: Examples of distortions in historical coins. 6 Conclusion The paper presented a reliable system for the classification of coins that are partially occluded. The system performs template matching based on gradient orientations in order to classify the coins. The system correctly classifies 76% of the coins in the 2007 MUSCLE CIS benchmark dataset, while making only 0.1% misclassifications. We presented an approach for occlusion detection based on texton-based texture classifiers, but found that occlusion detection does not improve the performance of the system. Future work is directed towards improving our system for the classification of Merovingian and Roman coins. The main issue that needs to be addressed is that the center of stamps of historical coins often does not coincide with the center of the coin itself. A possible way to address this issue is by allowing translations next to rotations in the similarity measure. Alternatively, the coin stamps could be registered by iteratively minimizing the distance between edge-based statistical features [9] before the similarity measure is computed. 1 The images are taken from http://freimore.uni-freiburg.de.
Acknowledgements This work was supported by NWO/CATCH under grant 640.002.401. References [1] A.J. Fitch, A. Kadyrov, W.J. Christmas, and J. Kittler. Orientation correlation. In Proceedings of the British Machine Vision Conference 2002, volume 1, pages 133 142, 2002. 4 [2] B. Fröba and C. Küllbeck. Orientation template matching for face localization in complex visual scenes. In International Conference on Image Processing 2000, pages 251 254, 2000. 4 [3] R. Huber, H. Ramoser, K. Mayer, H. Penz, and M. Rubik. Classification of coins using an eigenspace approach. Pattern Recognition Letters, 26(1):61 75, 2005. 1, 4 [4] J. Illingworth and J. Kittler. A survey of the Hough transform. Computer Vision, Graphics, and Image Processing, 44(1):87 116, 1988. 3 [5] M. Kampel and M. Zaharieva. Coins. In Proceedings of the CAA 2007 (to appear), 2007. 1 [6] T. Leung and J. Malik. Representing and recognizing the visual appearance of materials using three-dimensional textons. International Journal of Computer Vision, 43(1):29 44, 2001. 3 [7] M. Nölle. Distribution distance measures applied to 3D object recognition: A case study. In Proceedings of the 25 th Pattern Recognition Symposium of the German Association for Pattern Recognition, pages 84 91, 2003. 1, 4 [8] M. Reisert, O. Ronneberger, and H. Burkhardt. An efficient gradient based registration technique for coin recognition. In Proceedings of the MUSCLE CIS Coin Recognition Competition Workshop, pages 19 31, 2006. 1, 2, 4 [9] L.J.P. van der Maaten and P.J. Boon. Coin-o-matic: A new system for fast and reliable coin classfication. In Proceedings of the MUSCLE CIS Coin Recognition Competition Workshop, pages 7 17, 2006. 1, 2, 7 [10] L.J.P. van der Maaten, P.J. Boon, and E.O. Postma. Automatic classification of modern and historical coins. In Proceedings of the International Conference on Computer Vision 2007 (submitted), 2007. 1, 4, 5 [11] M. Varma and A. Zisserman. Classifying images of materials: Achieving viewpoint and illumination independence. In Proceedings of the 7 th European Conference on Computer Vision, volume 3, pages 255 271, 2002. 3