Evaluation of the storability of Piel de Sapo melons with sensor fusion

Transcription

1 Evaluation of the storability of Piel de Sapo melons with sensor fusion L. Lleó, P. Barreiro, A. Fernández, M. Bringas, B. Diezma and M. Ruiz-Altisent Physical Properties Laboratory, E.T.S.I.A., Polytechnic University of Madrid, Avda. Complutense s/n, 284 Madrid, Spain, Tel: Fax: Abstract Several varieties of melon have been evaluated under their storability point of view. Destructive (hollow volume, soluble solids, Magness-Taylor firmness) and non destructive measurements (impact firmness, acoustic response, multispectral features) have been carried out. Acoustic response shows a main variance in the range of Hz, decreasing when hollow volume and maturity increase. Multispectral images in chlorophyll band was selected as a suitable complement to acoustic frequency. Non supervised classification at harvest with multispectral camera is strongly correlated with acoustic frequency and impact acceleration. Fusion of acoustic response and multispectral classification allows to differentiate between internal hollows and maturity. Keywords: Melons, acoustic response, multispectral, sensor fusion, firmness, internal hollow, maturity INTRODUCTION Non destructive techniques exploiting the sonic characteristics of fruit tissue have been applied for firmness measurements as well as to internal disorders in several products such as apples, pears, avocados and melons. Frequently, instruments deliver an impulse to the fruit to produce acoustic vibration (Farabee and Stone, 1991; Armstrong et al., 1997; De Belie et al, 2). Different systems are used to sense the vibration of a fruit. Some instruments have piezo-electric sensors, while others employ microphones (De Baerdemaeker et al., 1982; Armstrong et al., 199; Stone et al., 1996; De Belie, et al., 2; Diezma et al. 23). Based on instrumental measurements, as well as on theoretical analysis, two fundamental mode shapes referred to as torsion modes and spherical modes have been found to exist for different fruits. Only some resonant frequency modes shapes has been related to fruit firmness. The experimental setup used in this research provides the resonant frequency of one spherical mode. It was reported by Stone et al. (1996) that in Galia melon the first-type spherical resonant frequency measured around the equator was between 23 and 28 Hz, which agrees well with the values of the peaks considered in our research. Former relations between this resonant frequency and melons firmness were established (Stone et al. 1996). Multispectral imaging may be used to address external features such as ripening (Lu, 24) and external defects with higher sensitivity compared to ordinary RGB imaging (Aleixos et al. 22, Leemans et al., 22, Kleynen et al., 24). The spectral bands used for this study were selected in a previous research works (Ruiz-Altisent et al., 2). The objective of this study is fuse the acoustic impulse response and multispectral images in order to predict the storability of individual piel de sapo melons within an online prospective. 523

2 MATERIALS AND METHODS Several varieties of melons Piel de Sapo ( Abran, Pinzón, MP-899, Seda, Nicolás, Valverde, Babiera, MP-857, MP-97, MP-91, Cantasapo, Ruidera, Trujillo, Montijo ), have been evaluated under their storability point of view with destructive and non destructive techniques. Two main experiments have been carried out. The first one aimed to address the maturity variability and internal quality at harvest and consisted of analyzing 135 melons evaluated for color, external hardness (impact), internal texture, hollow volume (missquality factor) and soluble solids (ºBrix) as reference parameters, while acoustic impulse response and multispectral images were used as non destructive procedures under sensor fusion strategy. The camera employed was a 3 CCD RGB (Red, Green and Blue) camera; each channel was centered in a specific wavelength: 66, 54, 46 nm respectively with a 4 nm of bandwidth in all cases. The second experiment (5 melons) was designed to evaluate mentioned non destructive techniques when used to predict the potential storability of melons. Melons were analyzed with mentioned non-destructive procedures at harvest and with both destructive and non destructive methods after one month storage at 2ºC. Also a set of 6 melons was analyzed with non destructive techniques four times along storage. In this case an Infrarred (IR), Red (R), Blue (B) camera was employed. (IR= 8 ±2nm, R=675±2nm, B=45±2nm). In this experiment, experts evaluated the melons and gave them a maturity score from 1, under ripe to 5 over ripe. RESULTS Acoustic response and internal hollow relationship Acoustic impulse response shows a main variance area in the range of Hz (see figure 2) which corresponds to the second vibration frequency. This vibration mode correlates with the hollow volume but also to over-ripening, (see figure 1 on the right). Multispectral images was selected as a suitable complement to address whether a frequency decrease is due to over-ripening or to internal hollow. 6 5 Hollow volume (ml) average peak freq. (Hz) Figure 1. Variability of internal hollow (left) and correlation between hollow volume and 2 nd vibration frequency (right) for the 15 melons corresponding to the initial experiment 524

3 Figure 2. Visualization of covariance matrix of acoustic spectra (Hz) considering the set of 6 melons along storage period (one month). The negative value of covariance is due to the displacement of the vibratory frequency to the left (lower value) as the period of storage increases. See also next figure amplitud (verde=dia, azul=dia1, rojo=dia3, negro=dia4 ) frecuencia Hz Hz amplitud (verde=dia, azul=dia1, rojo=dia3, negro=dia4 ) frecuencia Hz Hz Figure 3. Second vibration frequency (x-axis). Amplitude (y-axis) considering storage period; green: no storage, blue: two weeks, red: three weeks, black: four weeks. Post harvest evolution of two melons. Left melon was classified as storable though clear postharvest ripening is found. Right melon was classified as not storable and after 2 weeks of storage it had to be rejected due to unmarketable conditions. RGB and IR, R, B cameras 1. Maturity estimation The best relation between RGB images and expert evaluation was found in R channel. R histograms present a displacement to higher grey level values as the maturity score increases. At harvest, riper fruits reflect a higher amount of Red light while unripe ones are darker in that band, as expected for higher chlorophyll content. 525

4 RGB Camera. R channel (66 ± 4 nm) in this camera has wider wavelength range than R (675 ± 2 nm) for the IR, R, B camera. In both cases detector includes the chlorophyll peak absorption 675 nm. A non supervised classification based on Ward method is employed using all grey level from 3 to 24, which correspond to fruit segmentation thresholds compared to the background. Higher values than 24 correspond to very yellow coloured areas. Lower values than 3 are nearly constant. Two classifications were made independently for the bed side of images, and for the opposite side. The best results were obtained corresponding to bed images. Six natural grouped clusters were found in the population at harvest (135 melons from experiment 1). Three of them correspond to small fruits and the other three to large size melons, according to the size camera estimation (sum of pixels belonging to the fruit). In both size clusters, three maturity levels are found (fig 4). clu1-23 clu2-15 clu4-26 clu3-29 clu5-22 clu R 15 R 45 R 75 R 15 R 135 R 165 R 195 R R15 R45 R75 R15 R135 R165 R195 R225 Figure 4. Mean histogram for each non supervised category. On the left, big size groups: cluster 2 unripe, 1 medium, 4 ripe. On the right, small size clusters, 6 unripe, 5 medium, 3 ripe. The histograms move to the right when maturity increases. The number of fruits inside each cluster is indicated. Some relationships are found between the RGB classification and the acoustic response. When use resonant 2 Hz as threshold, fifty percent of big melons classified as non mature with the RGB camera show higher frequency. For medium, 25% of melons are above 2 Hz and only 1% for the ripest cluster. No tendency is observed in small size groups IR, R, B camera. Channel R 675 ± 2 nm, is narrower than R channel from R, G, B camera and we expect the images to be more related to chlorophyll degradation, and therefore to maturity. The described non supervised classification method was also applied. The grey level considered were from 15 to 15 which correspond to melon surface excluding lightest areas. Five categories from unripe to over ripe were found at harvest (19 melons) within storage experiment (5 melons in total). Again, the average histograms move towards 526

5 higher values. As cluster number increases, two regions seem to appear in the histogram. Two different populations appear inside the same image, inside the same fruit, probably corresponding to differences in chlorophyll content. Possibly due to a non homogeneous maturity process. 3 clu1-33 clu2-47 clu3-31 clu4-22 clu R15 R45 R75 R15 R135 Figure 6. Mean Red cluster histogram from 1, unripe to 5, over ripe. Bed images, 19 fruits. Red channel from IR, R, B camera. Experts score and camera classification present the same tendency. All fruits belonging to high categories (experts scores 4 and 5; clusters 4 and 5) present low firmness values. Comparing firmness (impact acceleration) with non supervised bed image classification, a clear tendency is found (see figure 6). Figure 6: Acoustic frequency (x-axis), maximum impact acceleration (m/s 2 ), and camera classification (no bed data, 19 fruits). Each point represents one fruit. As cluster score increases, the distribution of melons moves from high firmness (more than 7 m/s 2 ), high acoustic (above 2 Hz) to lower values. High maturity multiespectral classification (clusters 4 and 5) presents the whole range of frequency and lower firmness values 527

6 Figure 7. Acoustic frequency (x-axis), hollow volume (y axis), for each non supervised classification score. Cluster 1 presents the highest frequency and cluster 5 the lowest. Hollow volume is independent from multiespectral classification, and is negatively correlated with frequency. Camera classification is also negative correlated with frequency; riper fruits move to lower frequency. This relation is clearer than for the RGB camera. All fruits from cluster 1 present high frequency response. Few cases (3) in cluster 5 present frequency higher than 2 Hz. 2. Feature selection from histograms. Discriminant analysis. In both cameras the aim was to select the most discriminate variables, extracted from the Red histograms, to separate as much as possible each cluster from the others. Forward stepwise analysis was applied within this aim using cluster number as dependent variable. The independent variables, in the case of RGB camera, were grey levels from 4 to 14. Grey levels from 15 to 15 for IR, R, B. For the RGB camera the variables selected by mentioned procedure were level 6 (the most discriminative), 15 and 2. Considering only the first two, the percentage of correct classification was 91, 9 %. The IR, R, B camera was better as maturity classifier. In this camera, five maturity levels, and not only three could be segregated. Using 78 and 15 grey level the percentage of correct classification is 82,6%. When grouping fruits into three classes, the percentage of correct classification was 97,4 %. Figure 7, shows cluster classification with IR, R, B camera 528

7 clu1 clu2 clu3 clu4 clu5 f1yf2 f1yf3 f2yf3 f2yf4 f2yf5 f4yf5 f3yf r ,3% 96,5 % % 81 % 73 % r78 Figure 7. Representation of IR, R, B clusters and boundaries of discriminate functions (f1 to f5) with r78 and r15 (19 fruits). The maximum overlapping occurs between classes 1 and 2. Extreme clusters are completely separated one another. Arrows indicate the maturity evolution. CONCLUSIONS Maximum acoustic variance is found in the range of Hz. The second vibratory frequency correlates negatively with hollow volume and maturity. At harvest Red histograms present a displacement to the right as maturity increases. Camera non supervised classification is strongly correlated with frequency and firmness impact aceleration. This tendency is clearer in IR,R,B camera than in RGB. Expert and camera maturity classifications are correlated. Fusion of acoustic response allow to address whether frequency decrease, due to internal hollow or/and over ripening. CITATIONS Aleixos, N.; Blasco, J.; Navarrón, F. and Moltó, E. 22. Multispectral inspection of citrus in real-time using machine vision and digital signal processors. Computers and Electronics in Agriculture.33(2): Armstrong, P. R.; Zapp, H. R.; Brown, G. K Impulsive excitation of acoustic vibrations in apples for firmness determination. Transactions of the ASAE. 33:

8 Armstrong, P. R.; Stone, M. L.; Brusewitz, G. H Peach firmness determination using two different nondestructive vibrational sensing instruments. Transactions of the ASAE. 4: De Baerdemaeker, J.; Lemaitre, L.; Meire, R Quality detection by frequency spectrum analysis of the fruit impact force. Transactions of the ASAE. 25: De Belie, N.; Schotte, S.; Lammertyn, J.; Nicolai, B.; De Baerdemaeker, J. 2. Firmness changes of pear fruit before and after harvest with the acoustic impulse response technique. J. Agricultural Engineering Research. 77: Diezma-Iglesias, B.; Ruiz-Altisent, M. and Barreiro, P. 24. Detection of Internal Quality in Seedless Watermelon by Acoustic Impulse Response. Biosystems Engineering. 88(2): Farabee, M.; Stone, M. L Determination of watermelon maturity with sonic impulse testing. ASAE Meeting Presentation, paper Nº Kleynen, O.; Leemans, V., and Destain, M.-F. 23. Selection of the most efficient wavelength bands for Jonagold apple sorting. Postharvest Biology and Technology. 3(3): Leemans, V.; Magein, H., and Destain, M. -F. 22. AE Automation and Emerging Technologies: On-line Fruit Grading according to their External Quality using Machine Vision. Biosystems Engineering. 83(4): Lu, R. 24. Multispectral imaging for predicting firmness and soluble solids content of apple fruit. Postharvest Biology and Technology. 31(2): Ruiz-Altisent M.; Lleó L.; Riquelme F. 2. Instrumental quality assessment of fresh peaches: Optical and mechanical parameters. Proc. AgEng2, European Society of Agricultural Engineers Conference. Warwick, England 2-7 July. Stone, M. L.; Armstrong, P. R.; Zhang, X.; Brusewitz, G. H.; Chen, D. D Watermelon maturity determination in the field using acoustic impulse impedance techniques. Transactions of the ASAE. 39: ACKNOWLEDGES We thank the Syngenta Seeds for the economical support and the authorization to publish these results. FURTHER WORKS Defects camera detection, camera size estimation evaluation. Analysis of multispectral features from the other channels or combinations. Fusion of acoustic and multispectral analysis and classification applied to the whole storage period. Application of this methodology to another products. 53

9 Évaluation de la capacité de stockage des melons de Piel de Sapo par la fusion de sonde Mots-clés : melons, réponse acoustique, multispectrale, fusion de sonde, fermeté, cavité interne, maturité Résumé Plusieurs variétés de melon ont été évaluées selon leur capacité de stockage. Des mesures destructives (le volume de la cavité interne, les solides solubles, la fermeté de Magness-Taylor) et non destructives (la fermeté d'impact, la réponse acoustique, les dispositifs multispectraux) ont été effectuées. La réponse acoustique montre une grande variance sur une étendue de hertz, diminuant quand le volume de la cavité et la maturité augmente. Des images multispectrales étaient choisies dans la bande de chlorophylle comme un complément approprié à la fréquence acoustique. La classification non dirigée à la récolte avec un appareil photo multispectral est fortement corrélée avec la fréquence acoustique et l'accélération d'impact. La fusion de la réponse acoustique et de la classification multispectrale permet la différenciation des cavités internes et la maturité. 531

10 532