Constructing Distributed Fretted Instruments for the Web

Transcription

1 Generatve Art 22 Constructng Dstrbuted Fretted Instruments for the Web Prof. Jônatas Manzoll, BMath, BMus, MSc, PhD Insterdscplnary Nucleus for Sound Studes - NICS Musc Department, Arts Faculty, UNICAMP BRAZIL. e-mal:jonatas@ncs.uncamp.br Fernando Lndner Ramos, BCompEng Insterdscplnary Nucleus for Sound Studes - NICS Insttute of Computng, UNICAMP BRAZIL. e-mal:fernando@ncs.uncamp.br Frederck Carrlho, BMus Insterdscplnary Nucleus for Sound Studes - NICS Musc Department, Arts Faculty, UNICAMP BRAZIL. e-mal:fredcarlho@ncs.uncamp.br Abstract In ths artcle, we present a new computer based musc nstrument for dstrbuted performance on the Web. It was named Cordas Vrtuas and we took advantage of the recent Java2 mplementaton to create a general model for fretted-strng nstruments usng class abstractons. here s a heavy usage of JAVA Objected Orented nhertance to encapsulate gestures derved from fretted strng nstruments usng parameters such as strng tunng, group of strngs, hand gestures, rhythmc patterns, fngerng and alke, that carry each a set of attrbutes and propertes. We call ths knd of new muscal applcaton as Dstrbuted Muscal Instrument (DMI). As an applet, Cordas runs n any browser supportng the current Java Vrtual Machne (JVM) across the Web. We descrbe the concept of PlayStyle that was created to defne styles of fngerng on the strngs. In the mplementaton we have two classgroups: the left hand and the rght hand gestures. he left-hand classes control ptch changes or chords, and a rght hand control the rhythm, dynamcs, mcro-rhythms and rhythmc patterns. A Co-ordnaton Matrx controls real tme changes on left-hand movements. hs matrx generates chord orbts that are equvalent to the tradtonal chord cadences. Fnally, to show the potental of Cordas, we presented four muscal examples n whch a set of fretted nstruments varyng from the Classcal Gutar to the Chnese Ppa were smulated. Keywords: MIDI, Java2, nteractve musc, fretted nstruments, real tme 9.

2 Generatve Art 22. Introducton Recently, much research has been done to explore the muscal potental of the Internet: Burk [] developed a new Clent/Server archtecture for mult-user muscal performance, (see www. transjam.com). Helmuth [2] dscussed several host confguratons to allow musc performance on the Internet. Complementary, Hwang [3] dscussed the concept of Vrtual Muscal Envronment (VME) where muscans act n a mult-modal feedback process usng the Internet. Several new tools to explore the muscal potental of the Internet have been developed. Plenty of examples of these systems can be found n (see or the Java Musc Projects Beyond usng the Internet as a new muscal meda, the research presented here s n lne wth prevous works n whch "Interactve Musc System (IMS) has been studed. Rowe [4] defned nteractve musc systems as those whose behavour changes n response to musc nput. We studed a way of expandng the noton of IMS usng generatve methodologes. It was started, few years ago, wth the defnton of Sound Functors [5], a mathematcal defnton for muscal constructon and organsaton. We explored Evolutonary Computaton to created the compostonal envronment named VOX POPULI [6]. Later, we explore the use of a robot as an nteractve actor n a composton system, ths was called ROBOSER [7]. Eventually, we also explore the use of new hardware as nterface for musc performance as presented n [8]. Recently, we partcpate of the creaton of "ADA: the Intelgent Space" that used nteractve soundscapes to communcate behavoural changes to the audence. hs was presented at the EXPO.2 the Swss Natonal Exhbton for 554. vstors. (see hs paper presents a research that uses the Internet to expand the concept of IMS. Here we explore the noton of muscal nteracton creatng a new envronment that reproduces gestures of a muscan when he/she plays a fretted nstrument. In our system, class abstractons of a muscal gesture are used as nput. It can be also used to mplement famly of fretted nstruments whch we call Vrtual Strngs Instruments [9]. As an example, t s not dffcult to construct a 2-strng nstrument, each one assocated to a dfferent MIDI program and performed by a 7-fnger hand or, eventually, to smulate an ancent fretted nstrument. As t s dscussed later, these features are possble graces n hertage of the class abstracton used n our approach. he recent Java2 provdes nterfaces and classes for I/O, sequencng, and synthess of MIDI data. We used ths API to mplement fretted nstrument characterstcs lke number of strngs and frets, strng tunngs and the rght hand rhythmc actons. Cordas Vrtuas' classes are easly exchanged among the Web usng and muscans can mprovse, performng each one a personal strng nstrument. In the next sectons we present our research n detals. We descrbe the project mplementaton concepts such as JAVA class abstracton, GUI and play style n the frst secton. It follows the defnton of a smple mathematcal model based on a co-ordnaton matrx to control the lefthand movements. Fnally, we dscussed the mplementaton of a set of fretted strng nstrument as Internet applet and ths graphcal nstrument can be used to verfy the muscal potental of our system. 9. 2

3 2. Project Implementaton Generatve Art 22 Our am was to study how JAVA class mplementaton can be used n order to create new knds of sonc nteractons. As present n [], we have been worked wth the concept of Dstrbuted Musc Instruments (DMI) to allow muscans to play together and mprovse on the Web. In lne wth [], we developed a Clent/Server archtecture where the JAVA based MIDI server mports several MIDI events over the Internet from clents and ths allow muscans to Jam on the Web. In ths way, ths tested nteractve archtecture allows human and machne to co-operate n dstrbuted muscal performance stuatons. Addtonally, for modelng the muscal gesture, we adopted the pont of vew of a rght-hand nstrumentalst. he left hand controls the fret choce whle the rght hand attacks the strngs. We used the tablature notaton to descrbe the left-hand actons. It allowed a spatal ndex of a musc sequence provdng a precse ndcaton of the fret and the strng to be played. he rght hand movements and rhythm were descrbed by what we called Play Style. hs s a class abstracton related to the nstrument temporal control. Mcro-rhythmc structures such as those found on the Spansh Rasgueado Style were mplemented wth success, see he man classes were dvded n Instrument and Performer packages. Instrument contans the Syntheszer, nstrument name (MIDI program), number of strngs (ndvdual or grouped strngs). Performer contans representatons of the ablature and the Play Style, see [] for detals about the classes mplementaton. 2. Strngs, Frets and Graphc Interface Cordas Vrtuas s based on a oolbox developed to enable mult-task MIDI stream control, see Costa & Manzoll []. Independent sequences of MIDI data are generated and managed by a Note Collector smultaneously. he Play Style nterface for mcro-rhythmc manpulaton (see Fg. ) conssts on a nteracton area for each strng, where the horzontal axs determnes the perceptual duraton (the moment where the fnger touches the strng), and the vertcal axs represents the attack ntensty (fnger s velocty).. Fg. - Play Style Interface, vertcal lnes 9. 3

4 3. Generatng Chords Orbts 3. Co-ordnaton Matrx Generatve Art 22 Def.: We defned a Co-ordnaton Matrx, denoted as C n n, a square matrx that s used to control the co-ordnaton between the left an rght hand, n= number of strngs group and ts entres are defned n the set {,}. As an example we present below the matrx C 6 6, t was appled n our experment to generate sequence of chords that can be played n a classcal gutar { } C C2 C3 C4 C5 C6 C2 C22 C23 C24 C25 C26 C 3 C32 C33 C34 C35 C36 C 6 6 = () C4 C42 C43 C44 C45 C46 C 5 C52 C53 C45 C55 C56 C6 C62 C63 C46 C56 C66 wherec,,, j n, wth n = number of strng group = 6, each group has one strng. j Def.2: We defne the ablature Vector denoted as x, an array contanng the fret number played on the th strng denoted as x = fret number ( n wth n = number of strngs ). hs defnton s equvalent to the tradtonal tablature of fretted strnged nstruments. Snce we are usng the classcal gutar as references, the order of the frets s: the hghest ptched strng ( ) s related to, and the lowest ptched strng ( ) s related to x. E4 x E2 6 x = ] [ x x2 x3 x4 x5 x6 wth entres x {,,,..., m }, where m = number of frets and t depends upon the nstrument we are usng,.e. for classcal gutar m=9. he entry - s used to mean the strng s not played and t s assgned by the user. he entry s used to mean an opened strng. (2) Def.3: Gven an ntal ablature Vector x and a Co-ordnaton Matrx C 6 6 we call x K + the next ablature Vector n a sequence { x, x, x2,..., x n }. hs sequence s called here as Chord Orbt or Cadence, and t s generated by the matrx product and a mod operaton, as defned below: xk + = C x K (3) 9. 4

5 where K Generatve Art 22 k C 6 6 x nd s the ablature Vector and s the Co-ordnaton Matrx. In order to confne the entres of the k + followng operaton: x = K + x n the set {,,..., m} [ x m, x mod m,..., x mod m] mod 2 Z m n we apply at t k-step the where mod operaton gves the correspondent for each entry. As an example of a cadence defned for a Classcal Guter and generated by the Co-ordnaton Matrx, we ntroduce an ntal ablature Vector related to a G 7 chord x = [ 2 3]. he Co-ordnaton Matrx that generates the next chord C/G, x = [ 2 3 3] s presented below: x = C x = Def. 4: Gven a ablature Vector x = [ x x2 x3... x n ] MIDI note number table defned n [...27] n for each strng, the note number y = where (4) (5) n = number of strngs, the and a strng tunng note number wth played by Cordas Vrtuas s defned as: y = + x, n (6) where user. x {,,..., m} and t s generated by equaton (3) or the entry s assgned by the Usng ab., t s possble to calculate the MIDI note number played by the system. Gven the ablature presented n (4), the result s: [ ] = [ ] G 7 = [ ] (7) [ ] C/G = [ ] = [ ] (8) Classcal Gutar unng able strng st 2 nd 3 rd 4 th 5 th 6 th ptch E3 B2 G2 D2 A E MIDI Number ab. unng system of the classcal gutar. he frst row s the strng order, the second row contans the tunng for each strng and the thrd row s the MIDI note number. hese entres are equvalent to, wth

6 Generatve Art 22 Fg. 3 Cordas GUI Interface for a Classcal Gutar. In the left sde, the whte dots ndcate pressed fret postons. In the rght sde, dynamcs, mcro-rhythms and rhythmc patterns are descrbed by blue dot postons. 4. Instrument Case Studes As mentoned above, the Classcal Gutar was the frst nstrument studed, but the Cordas Vrtuas s objected orented structure allowed an easy creaton of new nstrument nstances. hs secton presents three applets mplemented usng Cordas, they are the Chnese Ppa, Andno Charango and the Afrcan Kora. he follow nformaton are needed: a) unng able and MIDI program number as seen on ab., b) Number of strng groups and c) Number of strngs per group. he mplementaton of the Kora applet requred to use to playstyle nterfaces, snce there s no fret use n ths nstrument and both hands are used to stroke the strngs (lke a harp). So, t was decded to use two PlayStyle pads, one n the rght sde and other n the left sde (see 4. Ppa Fg. 4 he Chnese Ppa wth four strngs It s a four-strnged Gutar lke nstrument, one of the oldest Chnese muscal nstruments created n Chna more than 2 years ago. he nstrument later was developed from ts orgnal two strngs desgn nto a form of four strngs and twelve frets, plucked wth fngernals and known as ppa or qn-ppa. Ppa was a general term referrng to those plucked-strng nstruments played n hand-held postons wth the outward fngerng technque called "p" and the nward one called "pa". Ppa unng able st strng 2 nd strng 3 rd strng 4 th strng A4 D3 E3 A ab 2. - unng system of the Chnese Ppa. 9. 6

7 Generatve Art Charango It s the man strnged nstrument used n Andean musc, he resonance box s usually made from an Armadllo shell or carved wood mtatng the shell. It s used n almost tradtonal musc of Peru and Bolva. Fg. 5 In the left, the Andean Charango wth 5 group of double strngs. Above, the GUI of Charango applet where one can see the double strngs on the left sde. Charango unng able st double strng 2 nd double strng 3 rd octave strng 4 th double strng 5 th double strng E5,E5 A4, A4 E4, E5 C4,C4 G3, G3 88,88 8,8 76,88 72,72 67,67 ab 3. unng system of the Andean Charango 4.2 Kora he Kora s arguably the most complex chordophone of Afrca. It s played n the westernmost part of Afrca n Mal, Gamba, Burkna Faso, Gunea, Serra Leone, and Senegal. he calabash s covered wth a cowhde that s stretched over the open sde of the half calabash and then left n the sun to dry tght and hold the handposts n place. A radtonal Kora has 2 strngs and t s not a fretted nstrument, actually Kora s played wth two hands smultaneously. Despte, t s not an nstrument we decded to mplement a new applet usng two Play Style pads, one for each hand. he left pad has strngs and the rght has. Fg. 6 he Afrcan Kora Kora Left Hand unng able B4 G4 E4 C4 A3 F3 D3 B2 A2 G2 C ab 4. he tunng system of the Kora s Left Hand 9. 7

8 Generatve Art 22 Kora Rght Hand unng able E5 D5 C5 A4 F4 D4 B3 G3 E3 C ab 5. - he tunng system of the Kora s Rght Hand Fg. 3 (left) Kora unnng system, (rght) Cordas Vrtuas Interface for Kora 5. Dscusson and Feather Developments We presented examples of four fretted strng applets: the Classcal gutar, the Chnese ppa, the Andean Charango and the Afrcan Kora. hese applets can be found n offcal Cordas Home Page ( he nstrument case studes presented above showed the potental of the software and ts flexblty resultng of the usage of class abstracton and nhertance. he mathematcal model used to defne the Co-ordnaton Matrx, s smple enough to be explored a dynamc system. he patterns on the Matrx can be controlled by several generatve algorthms. Partcularly, we ntend to apply evolutonary computaton as we have been studed n prevous works [6]. One of the most nterestng features of Cordas s that users can be seen as muscan performng a jam secton each one usng a dfferent nstrument applet. We have tested ths stuaton usng a Clent/Server archtecture we developed [,], the result was really an nterestng sound on the MIDI server. It produced a complex nterweavng of melodc lnes such as a Internet chat stuaton n whch a group users wrte messages n dfferent languages and the screen presents a multlngustc dscourse. he next steps of ths research wll be: ) o study mathematcal transformaton on the co-ordnaton matrx n order to dentfy and control patterns as well generatve structures; 2) o create an applcaton verson of Cordas mprovng several muscal and control parameters parameters of the GUI, such as Load, Save PlayStyle, Save MIDI Fle, Record Pattern; 3) o ntegrate Cordas wth other compostonal envronment we have developed at NICS, such as Roboser, VoxPopul, etc. Cordas wll be appled to provde the exchange of fretted nstrument features and performance characterstcs dstrbuted on the web. A database of tablature and Play Style wll be constructed n near future. 9. 8

9 Generatve Art References [] BURK, PHILIP L. 2. Jammn'on the Web - a new Clent/Server Archtecture for Mult-User Muscal Performance.Vsual. In Proceedngs of ICMC 2, Berln, Ed. he Internatonal Computer Musc Assocaton, 7-2. [2] HELMUH, M., 2. Sound Exchange and Performance on Internet2. In Proceedngs of ICMC 2, Berln, Ed. he Internatonal Computer Musc Assocaton, [3] HWANG, J., KIM, G. J., 2. Desgn and Analyss of Vrtual Muscal Envronments (VME). In Proceedngs of ICMC 2, Berln, Ed. he Internatonal Computer Musc Assocaton, [4] ROWE, R., 993. Interactve Muscal Systems. MI Press: Cambrdge Massachussets. [5] MANZOLLI, J. & MAIA, JR. A., 998. "Sound Functors Applcatons". In proceedngs of the V Brazlan Symposum on Computer Musc, XVII Congress of the SBC, UFMG, Belo Horzonte, Brazl, pg: 5-2. [6] MORONI, A., MANZOLLI, J., VON ZUBEN, F. & GUDWIN, R., 2, "Vox Popul: An Interactve Evolutonary Syste for Algorthmc Musc Composton", San Francsco, USA: Leonardo Musc Journal - MI Press, Vol.. [7] WASSERMANN, K. C., BLANCHARD, M., BERNARDE, U., MANZOLLI, J., AND VERSCHURE, P. F. M. J. 2. Roboser - An Autonomous Interactve Muscal Composton System. In Proceedngs of ICMC 2, Berln, Ed. he Internatonal Computer Musc Assocaton, [8] MORONI, A., MAMMANA, A. & MANZOLLI, J., 2. "InstrumentALL: a Vrtual Instrument". Proceedngs of the 4th Internatonal Conference on Generatve Art, Poltecnco d Mlano Unversty, Mlan, Italy. [9] COSA, M., MANZOLLI J. & SHARONI, D., 22. "A Dstrbuted Interactve Composton ool", SISGGRAPH, San Antono, USA, pg: 298 [] MANZOLLI, J., COSA M., RAMOS F., FORNARI, J. & SHARONI, D., 22. "Solutons for Dstrbuted Muscal Instruments on the Web", Procedngs of the Inagural Conference on the Prncples and Practce of Programmng n Java, rnty College, Dubln. pg: [] COSA, M. O., AND MANZOLLI, J., 2. oolbox para Aplcações Muscas na Internet. In Proceedngs of SBC&M 2, Recfe, Brazl,