Image Feature Extraction and Recognition of Abstractionism and Realism Style of Indonesian Paintings

Transcription

1 Image Feature Extraction and Recognition of Abstractionism and Realism Style of Indonesian Paintings Tieta Antaresti R P and Aniati Murni Arymurty Faculty of Computer Science University of Indonesia Depok 16, Indonesia tieta.antaresti@ui.ac.id, aniati@cs.ui.ac.id Abstract Tis paper cooses and evaluates tree feature vectors and teir augmented feature vector for recognizing te styles of Indonesian paintings. Te tree feature extraction metods include te Gabor wavelet, istogram analysis, and number-ofedge analysis. Te recognition purpose is to discriminate between te abstractionism and te realism styles of Indonesian paintings. Te experimental results using 115 painting images sow tat te use of number-of-edge features as given te best result wit 66.3% accuracy. Keywords visual arts, Indonesian paintings, Gabor wavelet, feature extraction, istogram, Canny edge detection I. INTRODUCTION One of te implementation of image processing tecnology wic can be applied widely to preserve te cultural eritage is te classification of paintings based on teir painting style or based on teir painters. Te aim of suc classification is to avoid te subjectivity wic can be occured in a manual recognition. Anoter objective is to invent and to build an information system of Indonesian paintings or to recognize te painter of an unrecognized old painting. So, an effective metod was needed to classify paintings effectively and efficiently. In tis study, tree feature extraction metods are cosen. Te first one is te Gabor wavelet features. Jonson et al. in [5] ave used tis metod for distinguising van Gog and non-van Gog s paintings based on te impasto (tick brusstrokes) on its digitized paintings. In [5], tis metod was applied to identify te energy and coarseness of te impasto. We are interested to apply tis metod in our work because it ad given a sarp analysis of te texture extraction and a quite encouraging results. Te second metod is te istogram analysis. It ad been used on [3] to classify images from several classes of objects suc as landscape, building, face, and indoor object, in a content based image retrieval system. Sergyan in [3] as used tis metod to evaluate 5 aspects of image istogram: mean, variance, skewness, energy, and entropy. Te purpose of tis recognition is to categorized te rigt image in te rigt classes. Tis work ad given a good result wit 87% classification accuracy. Te tird metod is te number-of-edge analysis. Actually, Jiang et al. [] ad implemented suc metod called as edge size istogram (ESH) on teir experimen to identify traditional Cinese paintings (TCP). We only evaluated te number-of-edge because it ave less computation cost tan evaluated an ESH. From [], we know tat te edge feature was quite representative to classify some kind of paintings based on its style, so we implemented some edge analysis too in our work. Tis paper is organized as follows. In te Section, te implementation along wit te teoretical background is explained. Te result of te experiments is discussed in Section 3. Tis paper is closed by a concluding remark and future work in Section. II. DATA AND METHODOLOGY A. Data Te data used for experimental consist of 115 painting image data from 16 digitized paintings wic was divided into tree classes: paintings wit abstractionism style, paintings wit realism style, and paintings wit undecided style. Paintings wit undecided style were not used on tis experiment. Tis categorization was made based upon te painter s style and by te definition of eac painting style. Te typical example of tose paintings style can be seen in Fig. 1. 1

2 Tere are two kinds of image format used in te experiment, te raw images and te binary edge images. Te binary image was obtained by processing te raw image using te Canny edge detection and creating a binary edge image. (a) Figure 1. Typical images: (a) abstractionism style and (b) realism style (Source of image: courtesy of Indonesia National Gallery) Te data from eac class or category were divided again into training and testing data wit proportion 3:7. Tis proportion was usually cosen based on te degree of recognition difficulty. For ard problem or if te recognition accuracy is not good, ten te proportion of training and testing data can be increased from 3:7 to 5:5, etc. B. Experiment using Gabor wavelet Gabor function is a wave wic is modulated by Gaussian function [1]. Gabor wavelet transformation was produced by a convolution process wit te formula as follows: G x y I x s y t s t * mn (, ) (, ) mn(, ) s t were s and t are te variables wic represents te size of filter mask and Ψ * mn is complex conjugate number from function Ψ mn wic came from te dilatation and rotation from moter wavelet Ψ(x,y). 1 x y [ ( )] x y jw 1 ( x, y) e e () x y Gabor wavelet evaluates te energy of eac image from processed image dataset to figure out ow te intensity levels are distributed in different scale and orientation [5]. For te experiment four scales and six orientations are used. Te output features consist of means and standard deviations of te energy of all orientation and scales. So, te size of output feature vector is 8 elements. Fig. sows te overall structure of te output feature vector from te Gabor wavelet metod. Figure. Structure of te output feature vector using Gabor wavelet metod (b) (1) Te training process is to estimate te mean of all training feature vectors. Finally, two estimators for abstractionism and realism category are found. Furtermore, in te testing process, tese estimators are used to decide weter a testing image belongs to te abstractionism style or te realism style. Te similarity measure is computed using te Euclidean distance. If te feature vector of a testing image is closer to te abstractionism style mean feature vector, ten te testing image is classified as abstractionism category; oterwise is classified as realism category. Te Euclidean distance is formulated as follows: (3) In [5], Gabor wavelet was sucessfully detected te impasto of Vincent van Gog. Te aim of tat impasto detection is to deterimine te van Gog and non-van Gog ones. Altoug tis metod on [5] ad not give te perfect accuracy as desired, tis metod can recognized te texture feature on te painting s impasto very well. C. Experiment using istogram analysis Te aim of using te features is to obtain better feature vectors wic can represent te caracteristics of an image by considering statistical features of its pixels. Te istogram tells us someting about te distribution of te grey level. Tere are five aspects wic will be analyzed for tese features. Tose aspects are mean, entropy, variance, skewness, and kurtosis [3] of te image istogram. Te mean of an image istogram represents brigtness of tat image. Higer mean represents brigter image, and vice versa. Te mean is computed from te image istogram by using tis formula: f p( f ) () In tis experiment, te value of L ranges from 0 to 55. Te f represents te value of intensity level, and te p(f n ) is te first-order istogram probability wic comes from te result of dividing number of pixels in some intensity level wit overall number of pixel on te image.

3 Variance is te square of standard deviation wic sows us te contrast level of an image by considering te distribution of gray level on tat image. An image wit ig variance will ave a ig contrast and vice versa. Variance can be computed by using tis formula: ( f ) p( f) (5) Skewness represents te skew of an image istogram. A skew-to-te rigt istogram wit a tail spreads to te rigt will ave a positive skewness value, and vice versa. Skewness from an image istogram can be obtained by using tis formula: skew 1 f p f 3 ( ) ( ) 3 (6) Kurtosis sows about ow sarp is an image istogram. A normal distribution image istogram will ave a zero kurtosis value. If te value more positive, te istogram curve will be more pointed too. Kurtosis can be computed by using tis formula: kurtosis 1 f p f ( ) ( ) 3 n n (7) Image istogram entropy measures te sape irregularity of te istogram curve. If te image is a simple image tat as a good distribution on various intensity levels, te istogram of tat image will ave a small kurtosis value, and vice versa. Te five features were computed in one feature vector wic ave five elements. Fig. 3 sows te overall structure of te output feature vector from tis feature extraction metod: Mean n Entropy n Variance n Skewness n Kurtosis n Figure 3. Structure of te output feature vector from using istogram analysis metod Te computation of te estimator of eac painting style category during te training process and te recognition during te testing pase based on te Euclidean distance, are done similarly to te use of te Gabor feature vector explained in sub-section B. In [3], tis metod was applied to recognize images from classes: landscape, building, face, and indoor objects in a CBIRS. Tese kind of images are te images wic usually used as painting objects. Tis metod ad sucessfully classified tese images wit 87% accuracy. D. Experiment using number-of-edge analysis Te number of edge features is extracted from an edge image, so tat an edge detection process to te raw image sould firstly take in place. Canny edge detector [] is used to detect all of te edges wic contains of bot strong and weak edges in an image. Te computation of weak edges was did by tis algoritm by using te ysteresis tresolding on te known edge. Te feature extraction process count te number of edges by tracing all te pixels wic ave te value 1 in te output binary edge image. If suc pixel was found, ten trace te 8-neigbor of it. If in te 8-neigborood one or more pixels wit value 1 were found, ten add tat pixels to and update te center pixel in te edge list. Fig. will sows about te possibility of edge detected on tis experiment: Figure. Two example of 3x3 images wic ave (a) 3 edges, and (b) 1 single edge only In [], te variation of tis metod called ESH was implemented to distinguis traditional and nontraditional Cinese paintings. ESH can detect te brustrokes very well but its computational cost was very expensive, so we take only te essence of tat metod, wic is number-of-edge analysis. ESH can identify up to 90% paintings wit SVM classifier. E. Augmented feature vector Te augmented feature vector were created by concatenating and cropping te feature vector elements of te tree feature vectors mentioned in te sub-section B, C and D. Te aim of tis experiment is to finda possibility to get a better feature vector, especially if tose features are complementary. After te vectors was modified, its painting style was distinguised by measuring te Euclidean distance of traing feature vectors to estimators. Te feature vector cropping is done by eliminating all te elements wic ave a very little difference in te two estimators of te painting style. First, compute any feature vector for eac estimators. Te difference of eac feature element tat is smaller tan a tresold value will be ignored. As a result, a feature vector wit a lesser number of elements is used in te same training and testing process procedure. 3

4 Te augmented feature vector is done by concatenating up to tree feature vectors from different feature extraction metod. Tere will be 3 combinations: Gabor wavelet features and istogram features, Gabor wavelet features and te number-of-edge features, and te istogram features and te number-of-edge features. Eac augmented feature vector is furter used in te training and testing process procedure. III. EXPERIMENT RESULTS Te recognition accuracy of eac metod is measured by computing te ratio of te number of correctly recognized paintings and te number of all paintings in testing dataset. From te experimental result, it can be seen tat te vector cropping metod does not effectively increase te recognition rate. It is probably because te metod deletes te unsignificant elements in te vector, so tat based on te computed Euclidean distance, te effect of te cropped feature vector compared to te fullelement vector does not make any difference. Tere are a few paintings tat cannot be recognized because of its different caracteristics from te usual one. Mostly of paintings of te realism style ave omogeneous background, wile texture and edge features contained in teir objects. Tat is wy te painting image in Fig. 5a tat as so many brusstrokes as been recognized as a painting of te abstractionism category. On te oter and, te cubism style of te abstractionism style can also be recognized as realism style because it ave a clear sape, altoug te sape was undetermined as a specific object, as sown in Fig. 5b. TABLE I. RESULT OF THE PAINTING CLASSIFICATION Metod Recognized Error Accuracy Gabor wavelet wit raw images % Gabor wavelet wit binary images Histogram analysis % Number-of-edge analysis % wavelet and num.of edge analysis wavelet and istogram analysis Concatenation of istogram analysis and num.of edge analysis Gabor wavelet wit raw images Gabor wavelet wit binary images wavelet and num.of edge analysis wavelet and istogram analysis Concatenation of istogram analysis and num.of edge analysis % % % % Concatenation of all metods Concatenation of all metods (a) Figure 5. Typical images tat ave been wrongly recognized (a) realism category, and (b) abstractionism category (Source of image: courtesy of Indonesia National Gallery) IV. (b) CONLUSION AND FUTURE WORK Tis work as developed a metodology to recognize two categories of painting style (abstractionism and realism) for Indonesian paintings. Tree feature vectors and teir augmentation are used, including Gabor wavelet, istogram aspects, and number-of-edge features. Te experimental results sow tat te number-of-edge feature vector performs te best wit te recognition accuracy of 66.3%. On te oter and, te istogram feature vector performs te worst wit te recognition accuracy around 8%. Te istogram features may only be good for nonpainting images, wile te painting images require information tat can sow texture and brusstrokes features. Te number-of-edge features is effective because of te Canny edge detector can detect te weak edges, so tat te real sape of object in a painting was revealed well in its binary edge image.

5 Te possible future works include te recognition of te painters of Indonesian paintings tat ave unique caracteristics suc as te well known Affandi. Fig. 6 sows a typical image of Affandi s artwork. Figure 6. Barong Melis, one of te well known Affandi s painting (Source of image: courtesy of Indonesia National Gallery) REFERENCES [1] A. Eka, Image Information Mining Pada Aplikasi Remote Sensing, Universitas Indonesia. [] J. Canny, A Computational Approac to Edge Detection, IEEE Transactions on Pattern Analysis and Macine Intelligence, PAMI-8, p [3] Sz. Sergyan, Color Histogram Based Image Classification in Content Based Image Retrieval System, [6t International Symposium on Applied Macine Intelligence and Informatics]. [] S.Jiang, Q.Huang, Q.Ye, W.Cao, An Effective Metod To Detect And Categorize Digitized Traditional Cinese Paintings, Pattern Recognition Letters Vol.7, p [5] C.R.Jonson, Jr., E.Hendriks, I.J.Bereznoy, E.Brevdo, S.M.Huges, I.Daubecies, J.Li, E.Postma, J.Z.Wang, Image Processing for Artist Identification, Computerized Analysis of Vincent van Gog s Painting Brusstrokes, IEEE Signal Processing Magazine. 5