International Journal of Informative & Futuristic Research ISSN (Online):

Transcription

1 Reviewed Paper Volume 2 Issue 6 February 2015 International Journal of Informative & Futuristic Research An Innovative Approach Towards Virtual Drums Paper ID IJIFR/ V2/ E6/ 021 Page No Subject Area Key Words Drum, Rock Band, Prototype, Thresholding, Matlab Electronics & Telecommunication Tandale Devidas 1 Vitnor Sagar B 2 Prof.Varsha Yanne 3 Nakate Ganesh 4 B.E. Student B.E. Student Professor B.E. Student Abstract Here we are proposing an approach towards virtual drum which can be used for any kind of entertainment. Also, it is having facility to increase the sound volume or stroke level. Matlab itself recognize the stroke and will play it accordingly. The percussion instruments are an important part of any musical composition and different cultures. The percussionists are responsible for holding the rhythm of the song and highlighting the important parts. To do this, they use different types of percussion elements like cymbals, kick drums, snares, tom drums, among others. Some of these elements are hit with long sticks at different times, to produce different sounds and set the rhythm of the composition. The elements of an ambient percussion kit can be replaced by rubber and plastic sensors that can simulate the same sounds. These electronic percussion kits are commonly used to digitize the interaction. Electronic drum sets allow manipulation of sound and strength with each element is hit. There are many attempts to simulate percussion. Copyright IJIFR

2 1. Introduction Since the twentieth century, teaching music has become a major activity for modern society. People who play an instrument or practice music have many cognitive benefits: improved linguistic capacity from singing, better bonding and cooperation with partners (e.g.: a band), increased physiological and motor skills and musical memory, among others. One of the main challenges is to encourage people to learn (how to play) music. Advances in natural interaction allow detection and recognition of spatial gestures, making possible the interaction with virtual objects. Learning how to play instruments can be encouraged through entertainment, with the creation of games with virtual musical instruments such as Rock Band or Guitar Hero. Using a depth capture device, mimicry of a user over virtual drums can be detected, and sounds can be played accordingly. This paper presents the development of a virtual drums prototype, based on natural interaction. The interaction between human and computerized musicians is an interesting concept to consider. This project will provide a computer simulation of a musical performance with a human drummer in an improvisational style. Not all musicians have immediate access to band members with whom they can practice and compose music with. This system would allow for individuals to see how their musical compositions will sound with a drummer who is able to follow along and adapt to changing musical conditions. In this paper we describe preliminary results of our research on virtual musicians. The objective of this project is to generate animated virtual musicians that play along with a given piece of music. There are many projects aiming to create a relationship between physical and digital workspaces. The Digital Desk provides an environment in which physical paper gains electronic properties, while physical objects can be used to operate electronic documents. There are also systems, such as Live Paper, that add digital information to enrich the interaction with physical artifacts like paper. Many research projects have specifically attempted to support architects and designers within the early stages of the design process. Electronic Paper explores augmented reality based approaches for supporting early stages of design. Luminous Table combines and integrates two dimensional drawings, physical models and digital models to support the urban design process. On the other side applications such as SKETCHPAD+, focus on creating 3D scenes in a natural and intuitive way on systems that only offer 2D interactions. Main concepts which are part of the theoretical framework of this work are presented the Natural Interaction as New User Interaction Paradigm Interaction, in the case of interactive software products, involves all kinds of exchange between a person (user) and a computer, for a specific task. Natural interaction focuses on the channel and the way this exchange is made, ensuring the use of natural language, movements, gestures and/or images. Natural interaction is possible through the use of several types of technology, such as multi-touch devices, augmented reality, voice recognition, and depth capture devices, among others. Some of the advantages of this type of (relatively new) interaction style are the naturalness, the ease of learning and the easiness of use. The latest may be particularly relevant for people with special needs or little computer knowledge [3] 2. Working Principle Here we are using USB webcam with 640 by 480 resolutions as source of input. Matlab code will trigger the camera and will capture the photo. This photo will be converted from RGB into HSV color space. Hue plane of HSV color space contains all the color information. So by putting some predefined thresholds we can segment some particular color of drum sticks Apex point of stick is 1604

3 located. As per the positions of the sticks, sound clip and corresponding stroke will be selected Code will be written in a Matlab in such a way that, it will generate corresponding sound and play it like drummer. Skin Filtering was performed to the input video frames for detection of hand gestures. It was done so that the required hand could be extracted from the background. Skin Filtering is a technique used for separating the skin colored regions from the non-skin colored regions. At first, the input frame was converted to HSV color space. This step was taken because HSV color space was less sensitive to illumination changes compared to RGB. Then it was filtered, smoothened and finally the biggest binary linked object was being considered so as to avoid consideration of skin colored objects other than hand. The resultant image is a binary image with hand regions in white and background in black color. The filtered hand is then found out. 3. Algorithm for Virtual Drum 1. Initialize camera. 2. Capture the image from camera. 3. Convert image from ycbcr format to RGBcolor format. 4. Convert RGB image into HSV image. 5. Extraction hue and saturation plane out forhsv image. 6. Segment out yellow plane by usingfollowingthresholding method.yellow = ((hue_plan<0.2)& (hue_plan>0.1) ) & ((sat_plane> 0.7)); 7. Find centroids of yellow colored heads ofdrum sticks. 8. Track motion of that centroid and store itinto array. 9. Find area of yellow ball. 10. Find whether its font back positiondepending upon area. 11. Also find speed of stroke by using rate ofchange of motion of drumstick. 12. Output the sound of various druminstruments depending upon front backposition and stroke rate of drum sticks. 13. Go to step

4 4. Flowchart Start Initialize Camera Capture Image image from ycbcr format to RGB&RGB to HSV yellow = ((hue plan<0.2)& (hue_plan>0.1) ) & ((sat_plane> 0.2)); Is centroid is yellow color No Track Motion Yes Is area of yellow ball found No Yes Give sound as per motion of stick End 1606

5 5. Result Program module hsv=rgb2hsv(im); hue_plan = hsv(:,:,1); sat_plane = hsv(:,:,2); yellow = ((hue_plan< 0.2)& (hue_plan>0.1) ) ); im_color = im; im_color(:,:,1) = uint8(yellow)*255; im_color(:,:,2) = uint8(yellow )*255; im_color(:,:,3) = 0; subplot(2,2,1); imshow( im); title('colorimage'); subplot(2,2,2); imshow( hsv); title('hsvimage'); subplot(2,2,3); imshow( yellow); title('yellowplane'); subplot(2,2,4); imshow( im_color); title('markers together'); Photo Shoot Picture 01 Picture 02 Picture

6 6. Application, Advantages & Disadvantages To save data written on white boards in the form of editable text. Helpful in education and virtual class rooms The Text recognized translated to other languages so that it can be read in any other desired language. Advantages Easy human machine interface. Saves the human energy though play like an original drum. Disadvantages Based on color recognition. May fail in bright sunlight or dark room. Poor response for night vision camera. 7. Conclusion The conclusion of this paper is to generate animated virtual musicians that play along with a given piece of music. Project will be extended to use it on a large scale i.e. in band, archeshtroor other such places. This is an just a concept based on virtual drums. References [1]A. Agarwal and B. Triggs, 3D human pose from silhouettes byrelevance vector regression, CVPR, pp , [2] Kölsch, M. and Turk, M. Keyboards without Keyboards: A Surveyof Virtual Keyboards, Workshop on Sensing and Input forn Media-centric Systems, Santa Barbara, CA, June 20-21, [3] D. H. Won, H. G. Lee, J. Y. Kim and J. Park, Development of AWearable Input Device Recognizing Human Hand and Finger Motions as A New Mobile Input Device, International Conferenceon Control, Automation and System, pp , 2001 [4] R. Rosales, V. Athitsos, L. Sigal, and S. Sclaroff, 3D hand posere construction using specialized mappings, ICCV, Vol. 1, pp , July [5] T. Darrell, I. Essa and A. Pentland, Task-Specific Gesture Analysis in Real-Time Using Interpolated Views, IEEE Trans. PAMI, vol. 18, no. 12, pp. 1,236-1,242, Dec [6] D. M. Gavrila and L. S. Davis, Towards 3D Model-Based Tracking and Recognition of Human Movement: A Multi-ViewApproach, AFGR, pp , June [7] Senseboard Tech. AB,