A MULTIDISCIPLINARY APPROACH TO THE CHARACTERI- ZATION OF BOWED STRING INSTRUMENTS: THE MUSICAL ACOUSTICS LAB IN CREMONA

Transcription

1 A MULTIDISCIPLINARY APPROACH TO THE CHARACTERI- ZATION OF BOWED STRING INSTRUMENTS: THE MUSICAL ACOUSTICS LAB IN CREMONA Roberto Corradi, Alessandro Liberatore, Stefano Miccoli Politecnico di Milano - Department of Mechanical Engineering, via La Masa 1, I-2156 Milano, Italy roberto.corradi@polimi.it Fabio Antonacci, Antonio Canclini, Augusto Sarti, Massimiliano Zanoni Politecnico di Milano - Dipartimento di Elettronica, Informazione e Bioingegneria, Piazza L. Da Vinci 32, I-2133 Milano, Italy The main purpose of this paper is to illustrate the research project of the Musical Acoustics Laboratory of Politecnico di Milano, recently established in the premises of the Violin Museum of Cremona. The research is focused on the characterization of bowed string instruments using methods from different disciplines: from material to vibrometric analysis and modelling, from acoustic radiance prediction to measurement and timbral analysis. This plethora of multimodal information is not just collected but jointly analysed and fused using machine intelligence methodologies. The techniques of analysis are described and some preliminary results are presented. 1. Introduction Modern bowed string instruments are the result of centuries of evolution based on empirical experimentation on the part of luthiers, aimed at improving acoustic projection and sound quality. Over the past century, several researchers began studying stringed instruments with scientific approach, by performing theoretical, experimental or numerical investigations. The old instruments of the great masters were analysed to examine material, design, and sound characteristics, using methods from acoustics, material science, structural mechanics, and computational science. Furthermore, relevant researches have been conducted on the vibrational behaviour of the soundboard, the interaction between the strings and the bow, the directivity pattern of acoustic radiation. However, the way instrument manufacturing influences the vibroacoustic behaviour and the timbral performance is still far from being fully understood. Moreover, the complexity of the involved physical and perceptive phenomena requires the contribution from different disciplines. This is the underlying idea of the Musical Acoustics Laboratory of Politecnico di Milano, recently established in the premises of the Violin Museum of Cremona with the purpose of investigating all the aspects of soundfield generation by stringed instruments. The related approach is truly multidisciplinary, from material to vibrometric analysis and modelling, from acoustic radiance prediction to measurement and timbral analysis. This plethora of multimodal information is not just collected but jointly analysed. ICSV22, Florence, Italy, July 215 1

2 In this paper the methodologies adopted to vibroacoustic characterization of violins are described. Experimental modal analysis was performed on the soundboard and the back plate of a contemporary violin at the most important stages of the manufacturing process, from the wood blocks to the completed violin. A numerical model of the instrument was developed starting from the geometry obtained with a 3D laser scanner. The collected experimental data are the starting point for setting up a vibroacoustic model of the violin, suitable for predicting the sound radiated by the instrument. In the Musical Acoustics Laboratory we are also investigating on novel methodologies for the non-invasive estimation of the radiance pattern of the instrument during a performance. At this purpose, techniques based on the space-time processing of audio signals are used. More specifically, a microphone array analyses the energy of the directional contributions of the soundfield emitted by the violin. A tracking system is used to determine the location and orientation of the instrument. This way, we are able to determine the portion of the radiance pattern that illuminates the microphone array at each time instant. In parallel with the vibroacoustic research, also the timbre of the instrument is analysed by means of methodologies coming from the machine learning literature. In summary, a corpus of sound acquired from historical and modern violins has been acquired. From each of these recordings, a number of synthetic descriptors are extracted. Examples of such descriptors are the centre of gravity of the spectrum, its spread, the ratio between the energies of the harmonic and non-harmonic components of the sound, etc. For each instrument, a panel of luthiers and violin players was asked to assess the timbral qualities of the violin according to a prescribed questionnaire. An analysis based on the correlation is then used to assess which descriptors are most relevant to determine the timbre. The final objective will be that of predicting the soundfield characteristics of bowed instruments from the mechanical behaviour of the numerical model of the soundboard, in order to provide support to the design of instruments. We are currently investigating on the use of machine learning to predict the timbral characteristics directly from the vibroacoustics acquisitions. 2. Experimental modal analysis 2.1 Experimental set up Experimental modal analysis tests were performed on the soundboard and the back plate at six steps of the manufacturing process. The violin, or the single components analysed, was suspending with nylon cables (Fig. 1) in order to decouple the flexible vibration modes from the rigid motion. Three piezoelectric uniaxial accelerometers (mass.5 g, sensitivity 1 mv/(m/s 2 ), range 9 m/s 2, bandwidth 1 khz) were placed on the soundboard. Both the soundboard and the back plate were excited using a small impact hammer (mass 5 g, sensitivity 22.5 mv/n) over 67 points in order to cover the entire surface. The positions of excitation and measurement are shown in Fig. 2. The Frequency Response Function (FRF) of the structure, was obtained as the averaged complex ratio between the acceleration of the jth point and the input force applied in the kth point from the experimental data. For each point of the measuring grid, at least 5 hits were averaged. Figure 2 shows the measurement grid and an example of experimental FRF obtained for i = 45 and j = 68, in terms of Mobility: the complex ratio between output velocity and input force. 2.2 Natural frequencies and mode shapes identification The Mobility FRF of Fig. 2 shows that up to about 2 khz the modes are clearly visible and it is possible to efficiently apply an identification procedure, based on a two-step approach, for the estimation of modal parameters from experimental FRFs. First, the Least Squares Complex Exponential (LSCE) algorithm [1, 2] is applied to estimate the system poles and the corresponding natural frequencies and damping ratios. The LSCE is a timedomain algorithm which requires in input the Impulse Response Functions (IRFs), the latter obtained 2 ICSV22, Florence, Italy, July 215

3 Figure 1: Experimental modal analysis set up: unvarnished violin with damped strings suspended with cables, accelerometers placed on the soundboard, impact hammer excitation on the back plate via inverse Fourier transform of the experimental FRFs, for all the considered j-k pairs. The second step consists in the classical FRF based curve fitting method (least squares minimization): according to the idea of modal superposition, the measured FRFs are fitted with an analytical model consisting of a series of second order systems. Least squares minimization operates in a user-defined frequency band which includes a certain number of vibration modes. To reach suitable curve-fit, high and low frequency residuals are added to the resonant modes [3], so as to account for the contribution of the modes not included in the selected frequency band. Figure 3 shows two examples of identified Mobility FRFs superimposed to the relative experimental FRFs. The corresponding first eight natural mode shapes of the free soundboard identified are reported in Fig. 4 (in agreement with [4, 5]). 1 Coherence.5 Mobility (db) Phase (deg) 9 9 (a) Measurement grid: excitation (black) and sensors (red) positions 18 Frequency (Hz) (b) Experimental Mobility FRF: free soundboard, excitation on node 45, response on node 68 Figure 2: Experimental FRF ICSV22, Florence, Italy, July 215 3

4 1 1 Experimental Identified 1 1 Experimental Identified Mobility (db) 3 5 Mobility (db) Phase (deg) 9 9 Phase (deg) Frequency (Hz) (a) Experimental (blue) and identified (dashed red) Mobility FRF: free soundboard, excitation on node 45, response on node Frequency (Hz) (b) Experimental (blue) and identified (dashed red) Mobility FRF: free soundboard, excitation on node 13, response on node 68 Figure 3: Comparison between experimental and identified Mobility FRFs of the free soundboard (a) Mode shape 1, f = 9.6 Hz (b) Mode shape 2, f = Hz (c) Mode shape 3, f = Hz (d) Mode shape 4, f = Hz (e) Mode shape 5, f = Hz (f) Mode shape 6, f = Hz (g) Mode shape 7, f = Hz (h) Mode shape 8, f = Hz Figure 4: Identified mode shapes of the free soundboard 4 ICSV22, Florence, Italy, July 215

5 3. Structural model A numerical model of the violin is implemented in order to simulate the structural behaviour of the instrument using the Finite Element Method (FEM). The development of the model proceeds in parallel to the experimental test, by reproducing the violin configurations at the same manufacturing process. The geometry of each component analysed is captured with a 3D laser scanner (6-axis arm length 2 m, tolerance.3 mm). A solid model, to be used for the FE analysis in order to simulate the major manufacturing step, is obtained by a reverse engineering process. The main purpose is to perform a step-by-step validation comparing the natural frequencies and mode shapes of the numerical model, extracted by an eigenvalue analysis, with the identified modal parameters for each stage. Thus it is possible to tune the model parameters in order to reach the maximum reliability and accuracy of the structural model of the complete violin. Moreover, it is possible to investigate the effect of each step of the manufacturing process on the vibroacoustic behaviour of the soundboard. Once the model is validated, it will be possible also to analyse the effect of different design solutions concerning the main components of the instrument. After this procedure has been done, the dynamic response of the model can be evaluated. The purpose is to predict the sound radiated by the violin using Boundary Element Method (BEM) analysis. 4. Acoustic radiance prediction An interesting sound property of the violin is its directivity or radiation pattern, which gives the angular dependency of the sound energy radiated from the instrument in the far field. The knowledge of the directivity allows us to infer how the sound will interact with the environment. The radiation pattern of the violin is a complex function that depends on several factors (e.g., materials and shapes), and it is difficult to predict, as shown in [6], and subtleties related to it can arise complex phenomena such as the directional tone color [7]. In Musical Acoustics Lab in Cremona we developed a novel technique for measuring the 3D radiation pattern of a violin. A compact rectangular microphone array (plenacoustic camera) is used to sense the directional components of the wave field. This is accomplished by the plenacoustic analysis of the wavefield proposed in [8], useful to estimate the directional acoustic energy at multiple points in space. This analysis is based on beamforming, and enables the measurement of the radiation pattern even in mildly reverberant rooms. A simple tracking system, composed of a depth map camera and a gyroscope, is used to track the position and the orientation of the musician with respect to the microphones. Each pose assumed by the violinist contributes to the portion of radiation pattern corresponding to the part of the instrument exposed to the plenacoustic camera. A reference microphone mounted in proximity of the violin is used to normalize the energy. Fig.5 summarizes the setup adopted for the estimation of the radiance pattern. Finally, Fig.6 shows the radiance pattern measured on a full-size student violin. The first row shows the radiation pattern on the horizontal plane (orthogonal to the ribs of the violin) measured in a controlled scenario (dashed line) or with the described methodology (solid line). The second row shows the full 3D radiation pattern. We show, in particular, the 5 highest harmonic peaks related to a D note at 294 Hz, up to 258 Hz. We observe that, at 294 Hz and 588 Hz, the radiation pattern is mostly omnidirectional, except for a slight energy damping in correspondence of the violinist s head and neck. As the frequency increases, the pattern becomes more directive and exhibits more irregular shapes. This behavior agrees with the predictions and results provided by Weinreich in [7], where the transition from a isotropic to an anisotropic radiation is found to occur approximately at 8 Hz. Indeed, at 1176 Hz there is a clear preferred direction of emission, while at 258 Hz the pattern presents three main radiation lobes. ICSV22, Florence, Italy, July 215 5

6 microphone array violin + gyroscope + mic depth map camera Figure 5: Setup of the system used to estimate the radiance pattern of the violin (a) Horiz. plane, f = 294 Hz (b) Horiz. plane, f = 588 Hz (c) Horiz. plane, f = 882 Hz (d) Horiz. plane, f = 1176 Hz (e) Horiz. plane, f = 258 Hz (f) Full pattern, f = 294 Hz (g) Full pattern, f = 588 Hz (h) Full pattern, f = 882 Hz (i) Full pattern, f = 1176 Hz (j) Full pattern, f = 258 Hz Figure 6: 3D radiation pattern of the violin under analysis: results in the horizontal plane are compared with the reference data (dashed line) obtained in a controlled scenario 5. Timbral analysis Timbral analysis refers to a set of techniques used to provide the characterization of the sound qualities of musical instruments. The study of the sound qualities of violins has been the subject of intense scientific investigation [9] for decades. However, the physical phenomena that are involved in the characterization of their timbral quality are still far from being fully understood [1]. In order to study the sound proprieties of musical instruments, one classical approach consists of extracting objective descriptors (Low-Level Features - LLF) [11] such as MPEG spectral and harmonic descriptors [12] or long term cepstral coefficients [13], and analysing how such descriptors cluster up in feature space. However, these descriptors are not semantically rich in nature, and do not match descriptions that are commonly used by violin makers and musicians (natural language - e.g. warm and bright), which are at a higher level of abstraction (Semantic Descriptors or High-Level Features - HLF) [1]. Though our way of describing sounds is based on subjective Semantic Descriptors, there exists a strong connection between sound description, sound perception and physics. Our brain, in 6 ICSV22, Florence, Italy, July 215

7 Distance Correlation Index rms zcr spectral centroid spectral spread spectral skewness spectral kurtosis rolloff spectral flux harmonic ratio spectral Irregularity mfcc1 mfcc2 mfcc3 mfcc4 mfcc5 mfcc6 mfcc7 mfcc8 mfcc9 mfcc1 mfcc11 mfcc12 mfcc13 attack time attack slope dark bright strident warm harsh sweet unfocused focused hard soft hoarse clean unbalanced balanaced weak strong thin full not deep deep nasal not nasal rounded not rounded Figure 7: Distance Correlation Index computed on Low-Level Features extracted from 11 violin recordings and semantic descriptors used for the annotations fact, processes stimuli from the auditory system in order to formulate a proper description. However, understanding what aspects of the sound influence our perception [1] is not an easy task. In Music Information Retrieval, an interdisciplinary research area that studies the extraction of useful information from musical content, the relation between objective timbral and acoustic proprieties (LLFs) and subjective semantic descriptors (HLFs) is studied by means of feature analysis methodologies [14]. In order to build the model for semantic descriptors we need to collect the low-level and the highlevel representations of a large set of instruments. As far as the low-level representation is concerned we recorded thirteen historical violins (three Amati, two Guarnieri del Gesú and eight Stradivari) and fifteen modern violins from the collection of the Museo del Violino in Cremona and International School of Lutherie (Stradivari Institute) in Cremona, played by a professional musician according to a specific protocol. For each recording we extracted a large set of LLFs selected in order to capture timbral and harmonic proprieties of the instrument. As far as HLFs are concerned, in [1] we collected and formalized the set of most relevant terms used in lutherie to describe the sound of violins. Furthermore, we collected the annotations by asking four professional violin makers to provide a description for each violin using the collected descriptors. In the listening test, each descriptor were presented along with its opposite (e.g. warm/not warm). The testers were asked to assign a graded annotation ranging from 1 to 1. Exploiting the low- and high-level sound characterization for each violin, we investigate the correlation between LLFs and HLFs through two well-known correlation methods [1]: Distance Correlation Index and a regression-based analysis using the RreliefF algorithm. Figure 7 shows the Distance Correlation Index for the LLFs and a subset of the semantic descriptors. It can be noticed that each semantic term has a strong correlation with the only a subset of audio cues. For example, Dark and Bright are intuitively mainly related to the predominance of low and high frequencies and this effect is confirmed by the Distance Correlation Index analysis. Figure 7, indeed, shows that Dark/Bright are mainly correlated with features that capture the spectral shape, such as Spectral Spread, Spectral Skewness and Spectral Kurtosis. Particularly important is also the high correlation with the mfcc3 and the mfcc13 features that are respectively related to low and high frequencies. Intuitively, semantic descriptors that share a semantic similarity, should also presents some similarity in the distribution of the correlation values with the LLFs. With the Distance Correlation Index analysis it is possible to capture this effect. For example, Fig. 7 shows how the terms Thin/Full ICSV22, Florence, Italy, July 215 7

8 and Rounded/Non Rounded presents a similar correlation with Spectral Centroid, Spectral Spread, Spectral Skewness, Spectral Kurtosis, Spectral Rolloff, mfcc1 and mfcc13 features. 6. Conclusions The research of the Musical Acoustics Laboratory of Politecnico di Milano is focused on the investigation of all the aspects of soundfield generation by bowed string instruments using a multidisciplinary approach. Experimental modal analysis was performed in order to validate a structural model of the violin to be used for predicting the dynamic response of the soundboard and the sound radiated by the instrument. The final objective will be that of predicting the timbral characteristics directly from the vibroacoustics acquisitions. 7. Acknowledgements This research activity has been partially funded by the Cultural District of the province of Cremona, Italy, a Fondazione CARIPLO project, and by the Arvedi-Buschini Foundation. The authors are grateful to the Violin Museum Foundation, Cremona, Italy, for supporting the acquisitions activities on historic violins, and Elena Bardella and Sebastiano Ferrari for providing the contemporary violin analysed. REFERENCES 1. W. Heylen, S. Lammens and P. Sas, Modal Analysis Theory and Testing, K.U.Leuven, (1998). 2. N.M. Maia and J.M. Silva, Theoretical and Experimental Modal Analysis, Wiley, New York, (1997). 3. D.J. Ewins, Modal Testing: Theory, Practice and Application, Wiley, 2nd ed., (21). 4. T.D. Rossing, The Science of String Instruments, Springer, (21). 5. C.M. Hutchins, The Acoustics of Violin Plates, Scientific American, Vol.245, No.4, (1981). 6. G. Bissinger, Some mechanical and acoustical consequences of the violin soundpost, The Journal of the Acoustical Society of America, (1995). 7. G. Weinreich, Directional Tone Color, The Journal of the Acoustical Society of America, (1997). 8. D. Markovic, F. Antonacci, A. Sarti and S. Tubaro, Soundfield Imaging in the Ray Space, IEEE Transactions on Audio, Speech and Language Processing, (213). 9. J. Woodhouse, The acoustics of the violin: a review, IOP Publishing, Reports on Progress in Physics, 77 (214). 1. M. Zanoni, F. Setragno and A. Sarti, The violin ontology In proceedings of the 9th Conference on Interdisciplinary Musicology (CIM14), Berlin, Germany, (214). 11. T. Sikora H.G. Kim, N. Moreau, MPEG-7 Audio and Beyond. Audio Content Indexing and Retrieval, John Wiley & Sons Ltd, (25). 12. A. Kaminiarz and E. Lukasik, Mpeg-7 audio spectrum basis as a signature of violin sound, In proceedings of the European Signal Processing Conference (EUSIPCO), (27). 13. E. Lukasik, Long term cepstral coefficients for violin identification, In proceedings of the Audio Engineering Society Convention 128 (AES128), (21). 14. E. Lukasik, Towards timbre-driven semantic retrieval of violins In Proceedings of the Fifth International Conference on Intelligent Systems Design and Applications (ISDA 25), (25). 8 ICSV22, Florence, Italy, July 215