Towards a Dynamic Model of the Palm Mute Guitar Technique Based on Capturing Pressure Profiles Between the Guitar Strings

Transcription

1 Proceedings ICMC SMC September 214, Athens, Greece Towards a Dynamic Model of the Palm Mute Guitar Technique Based on Capturing Pressure Profiles Between the Guitar Strings Julien Biral NUMEDIART Institute, University of Mons, Belgium julien.biral@gmail.com Nicolas d Alessandro NUMEDIART Institute, University of Mons, Belgium nda@numediart.org Adrian Freed CNMAT, University of California in Berkeley, USA adrian@cnmat.berkeley.edu ABSTRACT Electric guitar is one of the most emblematic musical instruments. It benefits from a large community that constantly extends its frontiers by innovating playing technique and adding control surfaces. This work focuses on palm muting, one of the most known electric guitar techniques. This technique is produced when the palm of the picking hand is used to damp the strings and this can have a strong effect on the timbre and dynamics of sound. However there are not known sensors or sound analysis techniques to precisely measure this effect. We introduce a new approach of sensing performance gestures by using pressure sensors between the strings. We explored several designs for the sensing system and have performed preliminary experiments on the relationship between the palm pressure, the sound and the behavior of the picking hand. 1. INTRODUCTION The electric guitar has become one of the most emblematic and ubiquitous musical instruments of popular culture. Amplification of string vibrations has also allowed the incorporation of what was once seen as extended techniques like bending, tapping, palm muting that are now fully assimilated as the regular practice of being an electric guitarist. Moreover, through the influence of guitarists like Jimmy Hendrix in the 7s, the electric guitar has become a field of sonic exploration, integrating tone research through additional effects in the performers skills, very similarly to what happened with keyboard performers and the analog (then digital, then computed-based) synthesizers. Most of guitar techniques quite straightforwardly follow Cadoz s classification of the musical gestures [1]: the fretting hand selects a note (or a group of notes) on the fretboard and the picking hand excites the string(s) to produce the sound; both hands can also be involved in the modulation of the guitar tone. Due to their production role, the gestures of the picking hand have a great influence on the attack, the intensity and the overall tone Copyright: c 214 Julien Biral et al. This is an open-access article distributed under the terms of the Creative Commons Attribution 3. Unported License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited. of the guitar sound. They are therefore very important. Among guitarists, it is often said that, while dealing with the fretboard fingering is an obvious thing to focus on, the subtlety in the picking reveals a greater level of mastery and musicality. Today one of the most popular technique is palm muting. It consists of damping the strings with the palm of the picking hand in the area near the guitar bridge. This paper focuses on palm muting: being able to recognize and parametrize this technique can lead to many improvements in various aspects of guitar playing: guitar transcription, guitar synthesizers/controllers and overall modeling of the picking hand. 1.1 Guitar Transcription Being able to automatically recognize techniques and gestures of guitarists is useful for guitar transcription. It is convenient to play a song while it is transcribed automatically on the computer, thanks to a software recognizing pitches and techniques. Building a sensor measuring palm muting is interesting because damping the string is a common technique to give other nuances to the sound. Moreover it has its own tablature notation. Figure 1. Diagram of the developed pressure sensor and its disposition: (1) Pickup; (2) Conductive band; (3) Fret (used as a sensor support); (4) Bridge; (5) Saddle

2 1.2 Guitar Synthesizers and Controllers Today plenty of guitar controllers are on the market. Some do not have strings [2]. Some scan the fingerboard to find exactly which notes are fretted [3]. Others use various techniques of pitch-detection, which have been improved considerably during the last decades. With the emergence of polyphonic pickups, pitch detection devices became more important and led to the development of systems directly connectable to the guitar. These systems allow the guitar to be used like a synthesizer, e.g. the Fishman Tripleplay and Roland VGuitar Systems. However those guitar controllers do not accurately track pitches when strings are palm muted. Therefore they generally require the performer to play without muting the strings [4]. 1.3 Modeling the Picking Hand Usually the fretting hand stops the guitar strings to establish the pitches of notes and chords that a guitarist plays. The picking hand is usually considered as the playing hand the one that has the most impact on timbre and timing. Much of the character of the personal style of the guitarist and the sound itself arise from the gestures of this playing hand. A fruitful axis of research is to model the relationship between the gestures of the picking hand and the sound produced so that gestures can be inferred from real-time analysis of the sound alone. This has been done successfully to identify the pick position [5, 6], the pick interaction [7] and the angle of attack [8]. Moreover multimodal recordings sound and sensors of the musical performances can help to bring this idea of surrogate sensing even further, by using machine learning techniques to determine the relation between audio and sensor data [9]. 1.4 Structure of the Paper In this paper Section 2 will present the overall arc of the work, i.e. the setup that we used and the building of the pressure sensor. The results will then be shown in Section 3 and discussed more deeply in Section 4. Finally we will conclude this work in Section SENSOR SETUP This section describes the fundamental concepts related to this work. Then we present the building of the pressure sensor: its position, its design and its refinement. Figure 1 illustrates the specific terminology related to the guitar field. 2.1 Guitar Techniques Proceedings ICMC SMC 214 The development of comprehensive models of guitar techniques is difficult because of the variety and the complexity of guitar sounds and the richness of contemporary performers explorations. An important contribution to this modeling problem was proposed by Reboursière et al. who used various classification approaches to recognize some popular guitar 14-2 September 214, Athens, Greece techniques [1]. This recognition was combined with real-time audio processing to create an augmented guitar. However their system can only recognize an ongoing technique as a whole, without further parametrization. Particularly the palm mute technique requires such a quantitive description because it is the vehicle with which many expressive changes to the sound are made including: shorter duration of the guitar note, attenuation of the global energy in the spectrum and less higher frequencies. These correspond to salient features for the listener such as loudness, brightness and attack time [11]. 2.2 Apparatus to Measure Palm Muting The apparatus we have built combines two measuring systems: a hexaphonic pickup for string vibration and a piezoresistive fabric, multipoint pressure sensor array Hexaphonic Pickups Among existing guitars with hexaphonic guitar pickups, we selected one that used piezoelectric sensors in the bridge saddles. This avoids potential problems associated with inductive coupling between current flows in the palm pressure sensor array and a magnetic pickup. The ability to record the six strings is important, as we can explore hypotheses such as how palm damping could reduce the crosstalk between strings that often complicates pitch tracking of guitars. It also allows us to temporally and spatially locate the primary source of energy driving the string from the initial pluck Pressure Sensing Array and Microcontroller Sensing palm pressure is challenging because it requires high spatial resolution, high sensitivity to light pressure, relatively high speed and no interference with the guitarist s regular playing technique. Piezoresistive fabric was chosen to solve the light pressure requirement and for its thinness and the ease with which it can be cut during prototype explorations. Commercially available force sensing resistors (FSR s) are simply not available in the required shapes and they are not effective for very light pressures. To confirm that fabric pressure sensing would work and to tune the choice of fabric, we built the first prototype using conductive strip board as shown in Figure 2. Conductors are wired alternately as signal input and ground to form an interdigitated linear array. Input signals were wired to a simple passive analog conditioning array, as illustrated in Figure 3, and the resulting voltages were acquired and translated into USB OSC messages using the Teensy 3.1 ARM microcontroller. We selected the Teensy because it is low cost, fast enough to send a sensing frame every 2 ms and it has an Arduino-compatible programming environment, including OSC library [12] support. The circuit shown in Figure 3 is a simple voltage divider in which the second resistor, R F ab, is implemented by the piezoresistive materiality of the fabric. This method is convenient in its simplicity and economy which is important because we replicate the circuit many times

3 Proceedings ICMC SMC September 214, Athens, Greece the hand and the pressure sensor. Indeed, the side of the hand has a complex structure which varies over time according to the gesture made by the guitarist. This is not an issue for our application as knowing the relationship of the textile to an applied force is sufficient for our purposes as we have ascertained that the functional relationship is monotonic [13]. Figure 2. First attempt to sense the guitar thanks to a pressure sensor array attached right onto the guitar bridge. Internal Resistance (Ohms) Force (N) Measured points Interpolated Curve V d + R F ab R C + V out Figure 3. Passive circuit of the pressure sensor. In this diagram, R F ab is the variable resistance of the fabric. The piezoresistive fabric was chosen with relatively low resistances to lower the thermal noise in the sensor. However, for a light touch, the impedance of the circuit is still too high to charge the storage capacitor of the A/D converter in the short time of the sampling operation. This problem is addressed with the additional capacitor which also serves to filter noise at high frequencies from nearby RF sources for example. In this work, a sampling frequency of 5 Hz has been achieved with a precision of 1 bits. Calibration of FSR s is notoriously difficult. In this application we used a lightweight Spandex from Eeonyx and take advantage of the good consistency of the fabric resistivity and the regular sensor spacing to produce repeatable relative values. Calibration measurements were made using the response of the internal resistance of the fabric to a force applied on the sensor, illustrated in Figure 4. The measured relationship between the force and the internal resistance of the textile is specially interesting for its sharp decreases for small forces, which allows for simple detection of very light contacts with the fabric. However, it is relatively difficult to convert this force into pressure due to the complex contacts between Figure 4. Measured response of the internal resistance of the piezoresistive fabric to an applied force on the pressure sensor. The sensor array was positioned immediately behind the bridge of a guitar and we were able to confirm that we could obtain good one-dimensional measurements of palm pressure. Unfortunately we found during musical performance that the significant measurements of palm pressure need to be done on the other side of the bridge, i.e. the side the strings vibrate on. In fact, there is nothing of interest to sense over the bridge itself. This makes the engineering problem much more challenging and perhaps explains why we could find no prior work on palm pressure sensing Saddle-Mounted Sensor Design The illustrations in Figure 1 and 6 show the challenging mechanical environment of the Gibson Les Paul bridge. The saddles are adjustable in position to refine pitch intonation and the whole bridge can be raised and lowered from the guitar top. We concluded that the palm sensing for each string had to be at the same points, adjacent to the string relative to the saddle position itself as the pressure of the side of the hand is applied on this area, as illustrated by Figure 5. This makes a single assembly impossible. Sensors are therefore integrated into the saddles themselves. In the first prototype of this idea, shown in Figure 8, we soldered fret wire to the saddles around which a series of conductive bands are wound (with appropriate heat shrink tubing for insulation). Piezoresistive fabric is then wrapped around this assembly. The series of bands affords two dimensional tactile imaging to reflect our observation that the rolling of the hand over the bridge creates a complex pressure profile along the first few centimeters of each string

4 Proceedings ICMC SMC 214 (a) 14-2 September 214, Athens, Greece (b) Figure 5. Positioning of the hand when the guitarist does palm muting: (a) palm muting area without the hand, (b) palm muting applied by the side of the hand. Therefore, the area between the bridge and the bridge pickup must be sensed to measure this technique. Figure 7. Equivalent electrical network of the pressure sensor, where Vd is the power voltage, the fabric, at the top, acts like variable resistors between each conductive ring. The resistors R are pull-up and pull-down resistors that allows the measure of the variable resistors and maintain a similar dynamic for each area. (a) Figure 6. Pressure sensor soldered on the saddle of a Les Paul bridge, sensing palm pressure along the string. (b) Figure 8. Prototype of one pressure sensor to mount along the string: (a) Profile view, (b) Front view. We see the fabric laying on the top of the conductive rings, whose are connected to wires Audio Recording Each area of the sensor is a derivative of a voltage divider, illustrated in Figure 7, where each conductive ring acts as an output and is connected to a voltage follower and a second-order Butterworth-like filter to prevent aliasing when the microcontroller samples the output voltages. This circuit has been analyzed and the variable resistances are inferred from the output voltages. The guitar audio is recorded from a Gibson Les Paul thanks to a hexaphonic pickup. The sound is amplified by the RMC Poly-Drive II and split in six jacks by the RMC Fanout Box. Finally, those six signals are gathered by the soundcard MOTU UltraLite MK3 and sent to the computer through the USB port. The audio can then be analyzed in Max/MSP or recorded for further examination. 2.3 Audio and Pressure Recordings 3. PRELIMINARY RESULTS This setup has been been developed to gather multimodal information about the palm muting technique. In this section, we explain the details of how we recorded these channels Pressure Gathering After being gathered by the sensors and the microcontroller, the pressure dataset is filtered digitally by a FIR filter with 128 coefficients, then it is downsampled, time-stamped and sent, via OSC packets, to the computer through the serial port. This dataset is then processed in the software Max/MSP and can be analyzed in real-time or recorded for further examination. Our custom pressure sensor has been tested in a simple playing situation, consisting of attacking the open E2 string first with no muting. Then the muting has been raised progressively until the guitar sounded heavily damped. Two different ways of palm muting have been tested: the former consists of moving the hand away from the bridge progressively, the latter consists of varying the applied pressure on the strings in a fixed position. Finally, three methods of plectrum picking are studied: the downward picking, the upward picking and the alternate picking. These tests have been recorded (audio and sensor data) and the following parts present some preliminary results computed in Matlab

5 Proceedings ICMC SMC September 214, Athens, Greece 3.1 Synchronization of the Datasets Normalized Pressure Intensity RMS Audio Envelope Normalized Pressure Normalized Magnitude First the pressure and audio datasets have been synchronized. Actually it is impossible to know automatically the delay between the two recordings because they are not sampled by the same processor. The pressure dataset is sampled by the ADC converter and time-stamped by the OSC protocol but the internal clock of the microcontroller is not the same as the one on the computer. The synchronization is thus made manually by hitting the E2 string at the beginning of the recording and realigning of the peaks in both datasets Time (s) (a) 1 Normalized Pressure Intensity RMS Audio Envelope Normalized Pressure The second test consists of rolling the palm along the sensor and measuring its response. This experiment is interesting because it illustrates the distribution of the pressure along the strings when the hand stands in different positions. The results are shown in Figure 9, where the time flows on the y-axis and where the x-axis corresponds to each four areas of the sensors (1 is the closest to the bridge and 4 is the closest to the bridge pickup). In this case the hand applied a strong pressure near the bridge (represented by the warmer colors on the graph) and slightly rolled off towards the bridge pickup until 44 seconds, before rolling back to the bridge. One can observe that the pressure applied on the first, second and fourth area of the sensor is more intense as the hand rolls off close to the pickup. However, the third area remains more or less constant and a less intense pressure is applied on it. This can be explained by the complexity of the side of the hand that applies the palm muting: the interaction between the location of muscles, the malleability of the skin and the distribution of the pressure that balances the hand on the guitar could explain this observation and further explorations are required to understand this phenomena Normalized Magnitude 3.2 Sensor Response to the Rolling of the Palm Time (s) (b) Normalized Pressure Intensity RMS Audio Envelope Normalized Pressure Normalized Magnitude Time (s) (c) Figure 1. Normalized pressure intensity over time on the second area of the sensor with its associated audio RMS envelope for: (a) downward picking, (b) upward picking, (c) alternate picking. 3.3 Picking Analysis Figure 9. Measured response of the sensor (x axis) to the rolling gesture of the palm over time (y axis). The intensity of the colors represents the intensity of the normalized pressure on the sensor. We performed pressure recording when a guitarist was playing with a plectrum with different picking techniques: downward picking, upward picking and alternate picking. The pressure profiles of three played notes as well as the RMS envelope of their audio signal are shown in Figure 1 for each kind of picking. We can see that a drop in intensity occurs every time the string is picked when palm muting. By looking at first at Figure 1a, we can

6 Proceedings ICMC SMC 214 divide each drop into three main phases: Just before the string is picked, the pressure intensity starts to fall slightly: this is probably due to the movement of the hand that anticipates the picking by positioning the fingers adequately. When the string is picked, the pressure intensity quickly falls to a lower value. Indeed, when guitarists play a string, they need to do it briefly with a certain force to produce a good sound. The hand can therefore be seen as a pivot where its applied force is distributed between the strain applied on the string and the movement required to play a note. After the string is picked, the pressure still continues to drop during a certain time. In fact, the hand keeps an inertia (momentum) after picking the string and then needs to be positioned to anticipate the next attack. The same phases are observable on the upward picking pressure set in Figure 1b and differentiating the two sets requires further analysis. However, the pressure data set of the alternate picking differs from the two first ones. Indeed, alternate picking is performed by strictly alternating downward and upward picking strokes in a continuous flow. This guitar technique is important because it is the most common method of plectrum playing. If the two first cases presented a similar behavior, the plot of the normalized pressure intensity over time of the alternate picking, shown in Figure 1c, is expected to give a slightly different result. Indeed, if in upward or downward picking, the hand must anticipate the gesture before the attack, alternate picking allows continuous attacks of the string without positioning the hand between each picking. One can observe that this is the case as each drop in pressure intensity is dividable in two phases: when the string is picked and after the string is picked. The phase that is preliminary to the attack does not appear in this plot. This informs our hypothesis that looking to the pressure profiles of a guitarist would be a way to recognize and study the playing of a guitarist. Moreover, as the sensor works in real-time, all this data could be interpreted and visualized directly. 4. DISCUSSION Our results clearly show that there is a correlation between the sound of the guitar and the behavior of the picking hand. The observation of the sensor values plotted together with the signal energy shows that there is an anticipation of the attack in the gesture made by the guitarist when he/she is palm-muting the string and picking in one direction (downward or upward). When the guitarist alternately picks the string, this anticipatory gesture seems to disappear as the hand strictly goes up and down to pick the string. These results mean that it is conceivable to know specifically the style of picking performed by a guitarist when palm muting in real-time, 14-2 September 214, Athens, Greece as the sensor is low-latency. This could lead to the rehabilitation of the playing of guitarists that suffer from injuries. Moreover, the study of the anticipatory gesture of the downward and upward picking is interesting as it could be a way to predict the attack of the string. Therefore, the observation of the sensor curves can give relevant information about the sound produced and the behavior of the hand, such as the intensity and the position of the pressure that produce a certain damped sound, or even knowing the strength of the attack by analyzing the steepness of the drop in pressure when picking. Finally the experiment consisting of rolling the hand along the sensor shows that the distribution of the pressure when palm muting is a complex problem that requires further work: damping heavily the string by rolling off the hand towards the bridge pickup does not simply translate the pressure intensity towards the pickup. The extraction of correlations between the intensity and the position of the applied pressure is consequently more difficult that has been imagined and necessitates a better understanding of the behavior of the hand motion of guitarists. 5. CONCLUSION In order to analyze the palm muting technique, we built a pressure sensor that can measure the pressure applied by the palm on the strings. Then we refined the sensor and explored the integration challenges of that kind of device on a popular guitar, a Gibson Les Paul. Our experiments clearly showed that it could sense the palm pressure accurately. Moreover the results that we obtained gave interesting information about the behavior of the picking hand, such as a slight release of the pressure before picking the string or that the distance of the hand from the bridge is more important than the applied force. These early experiments show us that this category of sensor will serve as an interesting platform for further research. Acknowledgments Thanks to the University of Mons and the AIMs for the sponsorship, to Gibson Guitars for the Les Paul, to John MacCallum for the help given about o.dot and Colleen McMahon for the drawings of Figures 1, 5, 7 and REFERENCES [1] C. Cadoz and M. M. Wanderley, Gesture-music, Trends in gestural control of music, vol. 12, p. 11, 2. [2] L. Kessous, J. Castet, and D. Arfib, GXtar, an interface using guitar techniques, in Proceedings of the 26 conference on New interfaces for musical expression. IRCAM Centre Pompidou, 26, pp [3] K. A. McMillen, Computer input device for polyphonic stringed instruments, in Proceedings of the International Computer Music Conference. Ann

7 Proceedings ICMC SMC September 214, Athens, Greece Arbor: 28. International Computer Music Association, [4] A. von dem Knesebeck and U. Zlzer, Comparison of pitch trackers for real-time guitar effects, in Proc. 13th Int. Conf. on Digital Audio Effects (DAFx), Graz, 21, pp [5] C. Traube and P. Depalle, Extraction of the excitation point location on a string using weighted least-square estimation of a comb filter delay, in Proceedings of the Conference on Digital Audio Effects (DAFx), 23. [6] H. Penttinen and V. Vlimki, A time-domain approach to estimating the plucking point of guitar tones obtained with an under-saddle pickup, Applied Acoustics, vol. 65, pp , 24. [7] F. Germain and G. Evangelista, Synthesis of guitar by digital waveguides: Modeling the plectrum in the physical interaction of the player with the instrument, in Applications of Signal Processing to Audio and Acoustics, 29. WASPAA 9. IEEE Workshop on. IEEE, 29, pp [8] B. Scherrer and P. Depalle, Extracting the angle of release from guitar tones: preliminary results, Acoustics 212 Nantes, 212. [9] A. Tindale, A. Kapur, and G. Tzanetakis, Training surrogate sensors in musical gesture acquisition systems, IEEE Transactions on Multimedia, vol. 13, pp. 5 59, Feb [1] L. Reboursire, O. Lhdeoja, T. Drugman, S. Dupont, C. Picard, and N. Riche, Left and right-hand guitar playing techniques detection, in Proc. of NIME, 212. [11] S. McAdams, Perspectives on the contribution of timbre to musical structure, Computer Music Journal, vol. 23, pp , [12] A. Schmeder, A. Freed, and D. Wessel, Best practices for open sound control, in Linux Audio Conference, Utrecht, NL, 1/5/ [13] D. Wessel, R. Avizienis, A. Freed, and M. Wright, A force sensitive multi-touch array supporting multiple 2- d musical control structures, in Proceedings of the 7th international conference on New interfaces for musical expression. ACM, 27, pp