Lecture 6: Nonspeech and Music
|
|
- Georgia Copeland
- 5 years ago
- Views:
Transcription
1 EE E682: Speech & Audio Processing & Recognition Lecture 6: Nonspeech and Music Music and nonspeech Environmental sounds Music synthesis techniques Sinewave synthesis Music analysis Dan Ellis <dpwe@ee.columbia.edu> Columbia University Dept. of Electrical Engineering Spring 26 E682 SAPR - Dan Ellis L6 - Nonspeech & Music
2 1 Music & nonspeech What is nonspeech? - according to research effort: a little music - in the world: most everything high speech music Information content low wind & water natural animal sounds attributes? contact/ collision Origin machines & engines man-made E682 SAPR - Dan Ellis L6 - Nonspeech & Music
3 Sound attributes Attributes suggest model parameters What do we notice about general sound? - psychophysics: pitch, loudness, timbre - bright/dull; sharp/soft; grating/soothing - sound is not abstract : tendency is to describe by source-events Ecological perspective - what matters about sound is what happened our percepts express this more-or-less directly E682 SAPR - Dan Ellis L6 - Nonspeech & Music
4 Motivations for modeling Describe/classify - cast sound into model because want to use the resulting parameters Store/transmit - model implicitly exploits limited structure of signal Resynthesize/modify - model separates out interesting parameters Sound Model parameter space E682 SAPR - Dan Ellis L6 - Nonspeech & Music
5 Analysis and synthesis Analysis is the converse of synthesis: Model / representation Synthesis Analysis Sound Can exist apart: - analysis for classification - synthesis of artificial sounds Often used together: - encoding/decoding of compressed formats - resynthesis based on analyses - analysis-by-synthesis E682 SAPR - Dan Ellis L6 - Nonspeech & Music
6 Outline Music and nonspeech Environmental sounds - Collision sounds - Sound textures Music synthesis techniques Sinewave synthesis Music analysis E682 SAPR - Dan Ellis L6 - Nonspeech & Music
7 2 Environmental Sounds Where sound comes from: mechanical interactions - contact / collisions - rubbing / scraping - ringing / vibrating Interest in environmental sounds - carry information about events around us.. including indirect hints - need to create them in virtual environments.. including soundtracks Approaches to synthesis - recording / sampling - synthesis algorithms E682 SAPR - Dan Ellis L6 - Nonspeech & Music
8 (from Gaver 1993) Collision sounds Factors influencing: - colliding bodies: size, material, damping - local properties at contact point (hardness) - energy of collision Source-filter model - source = excitation of collision event (energy, local properties at contact) - filter = resonance and radiation of energy (body properties) Variety of strike/scraping sounds - resonant freqs ~ size/shape - damping ~ material - HF content in excitation/strike ~ mallet, force t f E682 SAPR - Dan Ellis L6 - Nonspeech & Music
9 Sound textures What do we hear in: - a city street - a symphony orchestra How do we distinguish: - waterfall - rainfall - applause - static Applause4 Rain freq / Hz 3 2 freq / Hz time / s time / s Levels of ecological description... E682 SAPR - Dan Ellis L6 - Nonspeech & Music
10 Sound texture modeling (Athineos) Model broad spectral structure with LPC - could just resynthesize with noise Model fine temporal structure in residual with linear prediction in time domain y[n] Sound TD-LP y[n] = ia i y[n-i] Σ Per-frame spectral parameters e[n] Whitened residual DCT E[k] Residual spectrum - precise dual of LPC in frequency - poles model temporal events Temporal envelopes (4 poles, 256ms) FD-LP E[k] = Σib i E[k-i] Per-frame temporal envelope parameters amplitude time / sec Allows modification / synthesis? E682 SAPR - Dan Ellis L6 - Nonspeech & Music
11 Outline Music and nonspeech Environmental sounds Music synthesis techniques - Framework - Historical development Sinewave synthesis Music analysis elements? E682 SAPR - Dan Ellis L6 - Nonspeech & Music
12 3 Music synthesis techniques What is music? - could be anything flexible synthesis needed! Key elements of conventional music - instruments note-events (time, pitch, accent level) melody, harmony, rhythm - patterns of repetition & variation Synthesis framework: instruments: common framework for many notes score: sequence of (time, pitch, level) note events E682 SAPR - Dan Ellis L6 - Nonspeech & Music
13 The nature of musical instrument notes Characterized by instrument (register), note, loudness/emphasis, articulation... Frequency Piano Violin Frequency Time Clarinet Time Trumpet Time distinguish how? Time E682 SAPR - Dan Ellis L6 - Nonspeech & Music
14 Development of music synthesis Goals of music synthesis: - generate realistic / pleasant new notes - control / explore timbre (quality) Earliest computer systems in 196s (voice synthesis, algorithmic) Pure synthesis approaches: - 197s: Analog synths - 198s: FM (Stanford/Yamaha) - 199s: Physical modeling, hybrids Analysis-synthesis methods: - sampling / wavetables - sinusoid modeling - harmonics + noise (+ transients) others? E682 SAPR - Dan Ellis L6 - Nonspeech & Music
15 Analog synthesis The minimum to make an interesting sound Trigger Pitch + Vibrato + Cutoff freq Envelope t Oscillator t Filter f + Gain Sound Elements: - harmonics-rich oscillators - time-varying filters - time-varying envelope - modulation: low frequency + envelope-based Result: - time-varying spectrum, independent pitch E682 SAPR - Dan Ellis L6 - Nonspeech & Music
16 FM synthesis Fast frequency modulation sidebands: cos( ω c t + βsin( ω m t) ) phase modulation - a harmonic series if ω c = r ω m J n (β) is a Bessel function: = n = J n ( β) cos( ( ω c + nω m )t) 1 J J 1 J2 J 3 J4.5 J n (β) for β < n modulation index β Complex harmonic spectra by varying β 4 ω c ω m = 2Hz = 2Hz freq / Hz what use? time / s E682 SAPR - Dan Ellis L6 - Nonspeech & Music
17 Sampling synthesis Resynthesis from real notes vary pitch, duration, level Pitch: stretch (resample) waveform Hz 894 Hz time time / s time / s Duration: loop a sustain section time / s time / s Level: cross-fade different examples Soft mix veloc time / s time / s need to line up source samples Loud good & bad? E682 SAPR - Dan Ellis L6 - Nonspeech & Music
18 Outline Music and nonspeech Environmental sounds Music synthesis techniques Sinewave synthesis (detail) - Sinewave modeling - Sines + residual... Music analysis E682 SAPR - Dan Ellis L6 - Nonspeech & Music
19 4 Sinewave synthesis If patterns of harmonics are what matter, why not generate them all explicitly: sn [ ] = A k [ n] cos( k ω [ n] n) k - particularly powerful model for pitched signals Analysis (as with speech): - find peaks in STFT S[ω,n] & track - or track fundamental ω (harmonics / autoco) freq / Hz & sample STFT at k ω set of A k [n] to duplicate tone: mag time / s freq / Hz time / s Synthesis via bank of oscillators E682 SAPR - Dan Ellis L6 - Nonspeech & Music
20 Steps to sinewave modeling - 1 The underlying STFT: N 1 X[ k, n ] = xn+ n n = [ ] wn [ ] exp j 2πkn N What value for N (FFT length & window size)? What value for H (hop size: n = r H, r =, 1, 2...)? STFT window length determines freq. resol n: X w ( e jω ) = X( e jω ) W( e jω ) Choose N long enough to resolve harmonics 2-3x longest (lowest) fundamental period - e.g. 3-6 ms = khz - choose H N/2 N too long lost time resolution - limits sinusoid amplitude rate of change E682 SAPR - Dan Ellis L6 - Nonspeech & Music *
21 Steps to sinewave modeling - 2 level / db Choose candidate sinusoids at each time by picking peaks in each STFT frame: freq / Hz level / db Quadratic fit for peak: time / s freq / Hz y ab 2 /4 b/2 y = ax(x-b) x phase / rad freq / Hz freq / Hz + linear interpolation of unwrapped phase E682 SAPR - Dan Ellis L6 - Nonspeech & Music
22 Steps to sinewave modeling - 3 Which peaks to pick? Want true sinusoids, not noise fluctuations - prominence threshold above smoothed spec. level / db freq / Hz Sinusoids exhibit stability... - of amplitude in time - of phase derivative in time compare with adjacent time frames to test? E682 SAPR - Dan Ellis L6 - Nonspeech & Music
23 Steps to sinewave modeling - 4 Grow tracks by appending newly-found peaks to existing tracks: freq existing tracks birth death new peaks time - ambiguous assignments possible Unclaimed new peak - birth of new track - backtrack to find earliest trace? No continuation peak for existing track - death of track - or: reduce peak threshold for hysteresis E682 SAPR - Dan Ellis L6 - Nonspeech & Music
24 Resynthesis of sinewave models freq / Hz level After analysis, each track defines contours in frequency, amplitude f k [n], A k [n] (+ phase?) - use to drive a sinewave oscillators & sum up A k [n] n f k [n] A k [n] cos(2πf k [n] t) time / s time / s Regularize to exactly harmonic f k [n] = k f [n] freq / Hz time / s freq / Hz time / s what to do? E682 SAPR - Dan Ellis L6 - Nonspeech & Music
25 Modification in sinewave resynthesis Change duration by warping timebase - may want to keep onset unwarped 5 freq / Hz time / s Change pitch by scaling frequencies - either stretching or resampling envelope level / db freq / Hz level / db freq / Hz Change timbre by interpolating params E682 SAPR - Dan Ellis L6 - Nonspeech & Music
26 Sinusoids + residual Only prominent peaks became tracks - remainder of spectral energy was noisy? model residual energy with noise How to obtain non-harmonic spectrum? - zero-out spectrum near extracted peaks? - or: resynthesize (exactly) & subtract waveforms e s [ n] = sn [ ] A k [ n] cos( 2πn f k [ n] ) k mag / db must preserve phase! sinusoids original -6-8 LPC freq / Hz residual Can model residual signal with LPC flexible representation of noisy residual E682 SAPR - Dan Ellis L6 - Nonspeech & Music
27 Sinusoids + noise + transients Sound represented as sinusoids and noise: sn [ ] = A k [ n] cos( 2πn f k [ n] ) + k Sinusoids Parameters are {A k [n], f k [n]}, h n [n] h n [ n] bn [ ] * Residual e s [ n] freq / Hz time / s {A k [n], f k [n]} h n [n] Separate out abrupt transients in residual? e s [ n] = t k [ n] + h n [ n] b' [ n] k * - more specific more flexible E682 SAPR - Dan Ellis L6 - Nonspeech & Music
28 Outline Music and nonspeech Environmental sounds Music synthesis techniques Sinewave synthesis Music analysis - Instrument identification - Pitch tracking E682 SAPR - Dan Ellis L6 - Nonspeech & Music
29 5 Music analysis What might we want to get out of music? Instrument identification - different levels of specificity - registers within instruments Score recovery - transcribe the note sequence - extract the performance Ensemble performance - gestalts : chords, tone colors Broader timescales - phrasing & musical structure - artist / genre clustering and classification E682 SAPR - Dan Ellis L6 - Nonspeech & Music
30 Instrument identification Research looks for perceptual timbre space dull procedure? low attack bright low flux hi flux hi attack Cues to instrument identification - onset (rise time), sustain (brightness) Hierarchy of instrument families - strings / reeds / brass - optimize features at each level E682 SAPR - Dan Ellis L6 - Nonspeech & Music
31 Pitch tracking Fundamental frequency ( pitch) is a key attribute of musical sounds pitch tracking as a key technology Pitch tracking for speech - voice pitch & spectrum highly dynamic - speech is voiced and unvoiced ground truth? Applications - voice coders (excitation description) - harmonic modeling E682 SAPR - Dan Ellis L6 - Nonspeech & Music
32 Pitch tracking for music 4 Pitch in music - pitch is more stable (although vibrato) - but: multiple pitches Frequency 3 2 1?? Time Applications - harmonic modeling - music transcription ( storage, resynthesis) - source separation Approaches: place & time E682 SAPR - Dan Ellis L6 - Nonspeech & Music
33 Meddis & Hewitt pitch model Autocorrelation (time) based pitch extraction - fundamental period peak(s) in autocorrelation xt () xt ( + T) r xx ( T ) = xt ()xt ( + T) max Waveform x[n] time / samples Autocorrelation r xx [l] lag / samples Compute separately in each frequency band & summarize across (perceptual) channels Periodicity detection CF / Hz 4 Autocorrelogram sound Bandpass filters Rectification & low-pass filter Cross-channel sum Summary ACG lag / ms E682 SAPR - Dan Ellis L6 - Nonspeech & Music
34 sound Tolonen & Karjalainen simplification Multiple frequency channels can have different dominant pitches... But equalizing (flattening) the spectrum works: Prewhitening 1kHz 1kHz Rectify & low-pass Summary AC as a function of time: Periodogram for M/F voice mix f/hz 1 4 Periodicity detection Periodicity detection + SACF enhance ESACF Summary autocorrelation at t=.775 s time/s lag vs. freq? 2 Hz (.5s) 125 Hz (.8s) lag/s - Enhancement = cancel subharmonics E682 SAPR - Dan Ellis L6 - Nonspeech & Music
35 Post-processing of pitch tracks Remove outliers with median filtering 5-pt median Octave errors are common: - if x(t) x(t + T ) then x(t) x(t + 2T ) etc. dynamic programming/hmm time Validity - is there a pitch at this time? - voiced/unvoiced decision for speech Event detection - when does a pitch slide indicate a new note? E682 SAPR - Dan Ellis L6 - Nonspeech & Music
36 Summary Nonspeech audio - i.e. sound in general - characteristics: ecological Music synthesis - control of pitch, duration, loudness, articulation - evolution of techniques - sinusoids + noise + transients Music analysis - different aspects: instruments, pitches, performance and beyond? E682 SAPR - Dan Ellis L6 - Nonspeech & Music
Lecture 6: Nonspeech and Music. Music & nonspeech
EE E682: Speech & Audio Processing & Recognition Lecture 6: Nonspeech and Music 2 3 4 5 Music and nonspeech Environmental sounds Music synthesis techniques Sinewave synthesis Music analysis Dan Ellis
More informationLecture 6: Nonspeech and Music
EE E682: Speech & Audio Processing & Recognition Lecture 6: Nonspeech and Music 1 Music & nonspeech Dan Ellis Michael Mandel 2 Environmental Sounds Columbia
More informationLecture 5: Sinusoidal Modeling
ELEN E4896 MUSIC SIGNAL PROCESSING Lecture 5: Sinusoidal Modeling 1. Sinusoidal Modeling 2. Sinusoidal Analysis 3. Sinusoidal Synthesis & Modification 4. Noise Residual Dan Ellis Dept. Electrical Engineering,
More informationSound Synthesis Methods
Sound Synthesis Methods Matti Vihola, mvihola@cs.tut.fi 23rd August 2001 1 Objectives The objective of sound synthesis is to create sounds that are Musically interesting Preferably realistic (sounds like
More informationPreeti Rao 2 nd CompMusicWorkshop, Istanbul 2012
Preeti Rao 2 nd CompMusicWorkshop, Istanbul 2012 o Music signal characteristics o Perceptual attributes and acoustic properties o Signal representations for pitch detection o STFT o Sinusoidal model o
More informationE : Lecture 8 Source-Filter Processing. E : Lecture 8 Source-Filter Processing / 21
E85.267: Lecture 8 Source-Filter Processing E85.267: Lecture 8 Source-Filter Processing 21-4-1 1 / 21 Source-filter analysis/synthesis n f Spectral envelope Spectral envelope Analysis Source signal n 1
More informationPerception of pitch. Definitions. Why is pitch important? BSc Audiology/MSc SHS Psychoacoustics wk 4: 7 Feb A. Faulkner.
Perception of pitch BSc Audiology/MSc SHS Psychoacoustics wk 4: 7 Feb 2008. A. Faulkner. See Moore, BCJ Introduction to the Psychology of Hearing, Chapter 5. Or Plack CJ The Sense of Hearing Lawrence Erlbaum,
More informationSynthesis Techniques. Juan P Bello
Synthesis Techniques Juan P Bello Synthesis It implies the artificial construction of a complex body by combining its elements. Complex body: acoustic signal (sound) Elements: parameters and/or basic signals
More informationPerception of pitch. Definitions. Why is pitch important? BSc Audiology/MSc SHS Psychoacoustics wk 5: 12 Feb A. Faulkner.
Perception of pitch BSc Audiology/MSc SHS Psychoacoustics wk 5: 12 Feb 2009. A. Faulkner. See Moore, BCJ Introduction to the Psychology of Hearing, Chapter 5. Or Plack CJ The Sense of Hearing Lawrence
More informationspeech signal S(n). This involves a transformation of S(n) into another signal or a set of signals
16 3. SPEECH ANALYSIS 3.1 INTRODUCTION TO SPEECH ANALYSIS Many speech processing [22] applications exploits speech production and perception to accomplish speech analysis. By speech analysis we extract
More informationAdvanced audio analysis. Martin Gasser
Advanced audio analysis Martin Gasser Motivation Which methods are common in MIR research? How can we parameterize audio signals? Interesting dimensions of audio: Spectral/ time/melody structure, high
More informationSOUND SOURCE RECOGNITION AND MODELING
SOUND SOURCE RECOGNITION AND MODELING CASA seminar, summer 2000 Antti Eronen antti.eronen@tut.fi Contents: Basics of human sound source recognition Timbre Voice recognition Recognition of environmental
More informationPerception of pitch. Importance of pitch: 2. mother hemp horse. scold. Definitions. Why is pitch important? AUDL4007: 11 Feb A. Faulkner.
Perception of pitch AUDL4007: 11 Feb 2010. A. Faulkner. See Moore, BCJ Introduction to the Psychology of Hearing, Chapter 5. Or Plack CJ The Sense of Hearing Lawrence Erlbaum, 2005 Chapter 7 1 Definitions
More informationLecture 5: Speech modeling. The speech signal
EE E68: Speech & Audio Processing & Recognition Lecture 5: Speech modeling 1 3 4 5 Modeling speech signals Spectral and cepstral models Linear Predictive models (LPC) Other signal models Speech synthesis
More informationEE482: Digital Signal Processing Applications
Professor Brendan Morris, SEB 3216, brendan.morris@unlv.edu EE482: Digital Signal Processing Applications Spring 2014 TTh 14:30-15:45 CBC C222 Lecture 12 Speech Signal Processing 14/03/25 http://www.ee.unlv.edu/~b1morris/ee482/
More informationLecture 6: Speech modeling and synthesis
EE E682: Speech & Audio Processing & Recognition Lecture 6: Speech modeling and synthesis 1 2 3 4 5 Modeling speech signals Spectral and cepstral models Linear Predictive models (LPC) Other signal models
More informationL19: Prosodic modification of speech
L19: Prosodic modification of speech Time-domain pitch synchronous overlap add (TD-PSOLA) Linear-prediction PSOLA Frequency-domain PSOLA Sinusoidal models Harmonic + noise models STRAIGHT This lecture
More informationVIBRATO DETECTING ALGORITHM IN REAL TIME. Minhao Zhang, Xinzhao Liu. University of Rochester Department of Electrical and Computer Engineering
VIBRATO DETECTING ALGORITHM IN REAL TIME Minhao Zhang, Xinzhao Liu University of Rochester Department of Electrical and Computer Engineering ABSTRACT Vibrato is a fundamental expressive attribute in music,
More informationUniversity of Washington Department of Electrical Engineering Computer Speech Processing EE516 Winter 2005
University of Washington Department of Electrical Engineering Computer Speech Processing EE516 Winter 2005 Lecture 5 Slides Jan 26 th, 2005 Outline of Today s Lecture Announcements Filter-bank analysis
More informationBetween physics and perception signal models for high level audio processing. Axel Röbel. Analysis / synthesis team, IRCAM. DAFx 2010 iem Graz
Between physics and perception signal models for high level audio processing Axel Röbel Analysis / synthesis team, IRCAM DAFx 2010 iem Graz Overview Introduction High level control of signal transformation
More informationA Parametric Model for Spectral Sound Synthesis of Musical Sounds
A Parametric Model for Spectral Sound Synthesis of Musical Sounds Cornelia Kreutzer University of Limerick ECE Department Limerick, Ireland cornelia.kreutzer@ul.ie Jacqueline Walker University of Limerick
More informationSPEECH TO SINGING SYNTHESIS SYSTEM. Mingqing Yun, Yoon mo Yang, Yufei Zhang. Department of Electrical and Computer Engineering University of Rochester
SPEECH TO SINGING SYNTHESIS SYSTEM Mingqing Yun, Yoon mo Yang, Yufei Zhang Department of Electrical and Computer Engineering University of Rochester ABSTRACT This paper describes a speech-to-singing synthesis
More informationMUS421/EE367B Applications Lecture 9C: Time Scale Modification (TSM) and Frequency Scaling/Shifting
MUS421/EE367B Applications Lecture 9C: Time Scale Modification (TSM) and Frequency Scaling/Shifting Julius O. Smith III (jos@ccrma.stanford.edu) Center for Computer Research in Music and Acoustics (CCRMA)
More informationTempo and Beat Tracking
Lecture Music Processing Tempo and Beat Tracking Meinard Müller International Audio Laboratories Erlangen meinard.mueller@audiolabs-erlangen.de Book: Fundamentals of Music Processing Meinard Müller Fundamentals
More informationComplex Sounds. Reading: Yost Ch. 4
Complex Sounds Reading: Yost Ch. 4 Natural Sounds Most sounds in our everyday lives are not simple sinusoidal sounds, but are complex sounds, consisting of a sum of many sinusoids. The amplitude and frequency
More informationOverview of Code Excited Linear Predictive Coder
Overview of Code Excited Linear Predictive Coder Minal Mulye 1, Sonal Jagtap 2 1 PG Student, 2 Assistant Professor, Department of E&TC, Smt. Kashibai Navale College of Engg, Pune, India Abstract Advances
More informationDigital Speech Processing and Coding
ENEE408G Spring 2006 Lecture-2 Digital Speech Processing and Coding Spring 06 Instructor: Shihab Shamma Electrical & Computer Engineering University of Maryland, College Park http://www.ece.umd.edu/class/enee408g/
More informationINTRODUCTION TO COMPUTER MUSIC. Roger B. Dannenberg Professor of Computer Science, Art, and Music. Copyright by Roger B.
INTRODUCTION TO COMPUTER MUSIC FM SYNTHESIS A classic synthesis algorithm Roger B. Dannenberg Professor of Computer Science, Art, and Music ICM Week 4 Copyright 2002-2013 by Roger B. Dannenberg 1 Frequency
More informationSignal Analysis. Peak Detection. Envelope Follower (Amplitude detection) Music 270a: Signal Analysis
Signal Analysis Music 27a: Signal Analysis Tamara Smyth, trsmyth@ucsd.edu Department of Music, University of California, San Diego (UCSD November 23, 215 Some tools we may want to use to automate analysis
More informationLecture 6. Rhythm Analysis. (some slides are adapted from Zafar Rafii and some figures are from Meinard Mueller)
Lecture 6 Rhythm Analysis (some slides are adapted from Zafar Rafii and some figures are from Meinard Mueller) Definitions for Rhythm Analysis Rhythm: movement marked by the regulated succession of strong
More informationALTERNATING CURRENT (AC)
ALL ABOUT NOISE ALTERNATING CURRENT (AC) Any type of electrical transmission where the current repeatedly changes direction, and the voltage varies between maxima and minima. Therefore, any electrical
More informationNAME STUDENT # ELEC 484 Audio Signal Processing. Midterm Exam July Listening test
NAME STUDENT # ELEC 484 Audio Signal Processing Midterm Exam July 2008 CLOSED BOOK EXAM Time 1 hour Listening test Choose one of the digital audio effects for each sound example. Put only ONE mark in each
More informationCOM325 Computer Speech and Hearing
COM325 Computer Speech and Hearing Part III : Theories and Models of Pitch Perception Dr. Guy Brown Room 145 Regent Court Department of Computer Science University of Sheffield Email: g.brown@dcs.shef.ac.uk
More informationCS 591 S1 Midterm Exam
Name: CS 591 S1 Midterm Exam Spring 2017 You must complete 3 of problems 1 4, and then problem 5 is mandatory. Each problem is worth 25 points. Please leave blank, or draw an X through, or write Do Not
More informationTempo and Beat Tracking
Lecture Music Processing Tempo and Beat Tracking Meinard Müller International Audio Laboratories Erlangen meinard.mueller@audiolabs-erlangen.de Introduction Basic beat tracking task: Given an audio recording
More informationLinear Frequency Modulation (FM) Chirp Signal. Chirp Signal cont. CMPT 468: Lecture 7 Frequency Modulation (FM) Synthesis
Linear Frequency Modulation (FM) CMPT 468: Lecture 7 Frequency Modulation (FM) Synthesis Tamara Smyth, tamaras@cs.sfu.ca School of Computing Science, Simon Fraser University January 26, 29 Till now we
More informationFFT analysis in practice
FFT analysis in practice Perception & Multimedia Computing Lecture 13 Rebecca Fiebrink Lecturer, Department of Computing Goldsmiths, University of London 1 Last Week Review of complex numbers: rectangular
More informationCMPT 468: Frequency Modulation (FM) Synthesis
CMPT 468: Frequency Modulation (FM) Synthesis Tamara Smyth, tamaras@cs.sfu.ca School of Computing Science, Simon Fraser University October 6, 23 Linear Frequency Modulation (FM) Till now we ve seen signals
More informationProfessorial Inaugural Lecture, 26 April 2001 DIGITAL SYNTHESIS OF MUSICAL SOUNDS. B.T.G. Tan Department of Physics National University of Singapore
Professorial Inaugural Lecture, 26 April 2001 DIGITAL SYNTHESIS OF MUSICAL SOUNDS B.T.G. Tan Department of Physics National University of Singapore MUSICAL SYNTHESIS The many music synthesizers and keyboards
More informationSpeech Synthesis using Mel-Cepstral Coefficient Feature
Speech Synthesis using Mel-Cepstral Coefficient Feature By Lu Wang Senior Thesis in Electrical Engineering University of Illinois at Urbana-Champaign Advisor: Professor Mark Hasegawa-Johnson May 2018 Abstract
More informationINTRODUCTION TO COMPUTER MUSIC SAMPLING SYNTHESIS AND FILTERS. Professor of Computer Science, Art, and Music
INTRODUCTION TO COMPUTER MUSIC SAMPLING SYNTHESIS AND FILTERS Roger B. Dannenberg Professor of Computer Science, Art, and Music Copyright 2002-2013 by Roger B. Dannenberg 1 SAMPLING SYNTHESIS Synthesis
More informationConverting Speaking Voice into Singing Voice
Converting Speaking Voice into Singing Voice 1 st place of the Synthesis of Singing Challenge 2007: Vocal Conversion from Speaking to Singing Voice using STRAIGHT by Takeshi Saitou et al. 1 STRAIGHT Speech
More informationSingle Channel Speaker Segregation using Sinusoidal Residual Modeling
NCC 2009, January 16-18, IIT Guwahati 294 Single Channel Speaker Segregation using Sinusoidal Residual Modeling Rajesh M Hegde and A. Srinivas Dept. of Electrical Engineering Indian Institute of Technology
More informationApplications of Music Processing
Lecture Music Processing Applications of Music Processing Christian Dittmar International Audio Laboratories Erlangen christian.dittmar@audiolabs-erlangen.de Singing Voice Detection Important pre-requisite
More informationDigitalising sound. Sound Design for Moving Images. Overview of the audio digital recording and playback chain
Digitalising sound Overview of the audio digital recording and playback chain IAT-380 Sound Design 2 Sound Design for Moving Images Sound design for moving images can be divided into three domains: Speech:
More informationTHE CITADEL THE MILITARY COLLEGE OF SOUTH CAROLINA. Department of Electrical and Computer Engineering. ELEC 423 Digital Signal Processing
THE CITADEL THE MILITARY COLLEGE OF SOUTH CAROLINA Department of Electrical and Computer Engineering ELEC 423 Digital Signal Processing Project 2 Due date: November 12 th, 2013 I) Introduction In ELEC
More informationRhythmic Similarity -- a quick paper review. Presented by: Shi Yong March 15, 2007 Music Technology, McGill University
Rhythmic Similarity -- a quick paper review Presented by: Shi Yong March 15, 2007 Music Technology, McGill University Contents Introduction Three examples J. Foote 2001, 2002 J. Paulus 2002 S. Dixon 2004
More informationMusic Signal Processing
Tutorial Music Signal Processing Meinard Müller Saarland University and MPI Informatik meinard@mpi-inf.mpg.de Anssi Klapuri Queen Mary University of London anssi.klapuri@elec.qmul.ac.uk Overview Part I:
More informationEE 225D LECTURE ON MEDIUM AND HIGH RATE CODING. University of California Berkeley
University of California Berkeley College of Engineering Department of Electrical Engineering and Computer Sciences Professors : N.Morgan / B.Gold EE225D Spring,1999 Medium & High Rate Coding Lecture 26
More informationPhysics 101. Lecture 21 Doppler Effect Loudness Human Hearing Interference of Sound Waves Reflection & Refraction of Sound
Physics 101 Lecture 21 Doppler Effect Loudness Human Hearing Interference of Sound Waves Reflection & Refraction of Sound Quiz: Monday Oct. 18; Chaps. 16,17,18(as covered in class),19 CR/NC Deadline Oct.
More informationFinal Exam Study Guide: Introduction to Computer Music Course Staff April 24, 2015
Final Exam Study Guide: 15-322 Introduction to Computer Music Course Staff April 24, 2015 This document is intended to help you identify and master the main concepts of 15-322, which is also what we intend
More informationMusic 171: Amplitude Modulation
Music 7: Amplitude Modulation Tamara Smyth, trsmyth@ucsd.edu Department of Music, University of California, San Diego (UCSD) February 7, 9 Adding Sinusoids Recall that adding sinusoids of the same frequency
More informationMel Spectrum Analysis of Speech Recognition using Single Microphone
International Journal of Engineering Research in Electronics and Communication Mel Spectrum Analysis of Speech Recognition using Single Microphone [1] Lakshmi S.A, [2] Cholavendan M [1] PG Scholar, Sree
More informationStructure of Speech. Physical acoustics Time-domain representation Frequency domain representation Sound shaping
Structure of Speech Physical acoustics Time-domain representation Frequency domain representation Sound shaping Speech acoustics Source-Filter Theory Speech Source characteristics Speech Filter characteristics
More informationToward Automatic Transcription -- Pitch Tracking In Polyphonic Environment
Toward Automatic Transcription -- Pitch Tracking In Polyphonic Environment Term Project Presentation By: Keerthi C Nagaraj Dated: 30th April 2003 Outline Introduction Background problems in polyphonic
More informationWARPED FILTER DESIGN FOR THE BODY MODELING AND SOUND SYNTHESIS OF STRING INSTRUMENTS
NORDIC ACOUSTICAL MEETING 12-14 JUNE 1996 HELSINKI WARPED FILTER DESIGN FOR THE BODY MODELING AND SOUND SYNTHESIS OF STRING INSTRUMENTS Helsinki University of Technology Laboratory of Acoustics and Audio
More informationB.Tech III Year II Semester (R13) Regular & Supplementary Examinations May/June 2017 DIGITAL SIGNAL PROCESSING (Common to ECE and EIE)
Code: 13A04602 R13 B.Tech III Year II Semester (R13) Regular & Supplementary Examinations May/June 2017 (Common to ECE and EIE) PART A (Compulsory Question) 1 Answer the following: (10 X 02 = 20 Marks)
More informationSpeech Processing. Undergraduate course code: LASC10061 Postgraduate course code: LASC11065
Speech Processing Undergraduate course code: LASC10061 Postgraduate course code: LASC11065 All course materials and handouts are the same for both versions. Differences: credits (20 for UG, 10 for PG);
More informationSpeech Signal Analysis
Speech Signal Analysis Hiroshi Shimodaira and Steve Renals Automatic Speech Recognition ASR Lectures 2&3 14,18 January 216 ASR Lectures 2&3 Speech Signal Analysis 1 Overview Speech Signal Analysis for
More informationAdvanced Functions of Java-DSP for use in Electrical and Computer Engineering Senior Level Courses
Advanced Functions of Java-DSP for use in Electrical and Computer Engineering Senior Level Courses Andreas Spanias Robert Santucci Tushar Gupta Mohit Shah Karthikeyan Ramamurthy Topics This presentation
More informationSinging Voice Detection. Applications of Music Processing. Singing Voice Detection. Singing Voice Detection. Singing Voice Detection
Detection Lecture usic Processing Applications of usic Processing Christian Dittmar International Audio Laboratories Erlangen christian.dittmar@audiolabs-erlangen.de Important pre-requisite for: usic segmentation
More informationSince the advent of the sine wave oscillator
Advanced Distortion Analysis Methods Discover modern test equipment that has the memory and post-processing capability to analyze complex signals and ascertain real-world performance. By Dan Foley European
More informationAudio Engineering Society Convention Paper Presented at the 110th Convention 2001 May Amsterdam, The Netherlands
Audio Engineering Society Convention Paper Presented at the th Convention May 5 Amsterdam, The Netherlands This convention paper has been reproduced from the author's advance manuscript, without editing,
More informationSpeech Synthesis; Pitch Detection and Vocoders
Speech Synthesis; Pitch Detection and Vocoders Tai-Shih Chi ( 冀泰石 ) Department of Communication Engineering National Chiao Tung University May. 29, 2008 Speech Synthesis Basic components of the text-to-speech
More informationSGN Audio and Speech Processing
Introduction 1 Course goals Introduction 2 SGN 14006 Audio and Speech Processing Lectures, Fall 2014 Anssi Klapuri Tampere University of Technology! Learn basics of audio signal processing Basic operations
More informationEnhanced Waveform Interpolative Coding at 4 kbps
Enhanced Waveform Interpolative Coding at 4 kbps Oded Gottesman, and Allen Gersho Signal Compression Lab. University of California, Santa Barbara E-mail: [oded, gersho]@scl.ece.ucsb.edu Signal Compression
More informationLecture 9: Time & Pitch Scaling
ELEN E4896 MUSIC SIGNAL PROCESSING Lecture 9: Time & Pitch Scaling 1. Time Scale Modification (TSM) 2. Time-Domain Approaches 3. The Phase Vocoder 4. Sinusoidal Approach Dan Ellis Dept. Electrical Engineering,
More informationFormant Synthesis of Haegeum: A Sound Analysis/Synthesis System using Cpestral Envelope
Formant Synthesis of Haegeum: A Sound Analysis/Synthesis System using Cpestral Envelope Myeongsu Kang School of Computer Engineering and Information Technology Ulsan, South Korea ilmareboy@ulsan.ac.kr
More informationEffects of Reverberation on Pitch, Onset/Offset, and Binaural Cues
Effects of Reverberation on Pitch, Onset/Offset, and Binaural Cues DeLiang Wang Perception & Neurodynamics Lab The Ohio State University Outline of presentation Introduction Human performance Reverberation
More informationSignal Processing for Speech Applications - Part 2-1. Signal Processing For Speech Applications - Part 2
Signal Processing for Speech Applications - Part 2-1 Signal Processing For Speech Applications - Part 2 May 14, 2013 Signal Processing for Speech Applications - Part 2-2 References Huang et al., Chapter
More informationSpectrum. Additive Synthesis. Additive Synthesis Caveat. Music 270a: Modulation
Spectrum Music 7a: Modulation Tamara Smyth, trsmyth@ucsd.edu Department of Music, University of California, San Diego (UCSD) October 3, 7 When sinusoids of different frequencies are added together, the
More informationSound waves. septembre 2014 Audio signals and systems 1
Sound waves Sound is created by elastic vibrations or oscillations of particles in a particular medium. The vibrations are transmitted from particles to (neighbouring) particles: sound wave. Sound waves
More informationMusic 270a: Modulation
Music 7a: Modulation Tamara Smyth, trsmyth@ucsd.edu Department of Music, University of California, San Diego (UCSD) October 3, 7 Spectrum When sinusoids of different frequencies are added together, the
More informationSINUSOIDAL MODELING. EE6641 Analysis and Synthesis of Audio Signals. Yi-Wen Liu Nov 3, 2015
1 SINUSOIDAL MODELING EE6641 Analysis and Synthesis of Audio Signals Yi-Wen Liu Nov 3, 2015 2 Last time: Spectral Estimation Resolution Scenario: multiple peaks in the spectrum Choice of window type and
More informationSound is the human ear s perceived effect of pressure changes in the ambient air. Sound can be modeled as a function of time.
2. Physical sound 2.1 What is sound? Sound is the human ear s perceived effect of pressure changes in the ambient air. Sound can be modeled as a function of time. Figure 2.1: A 0.56-second audio clip of
More informationFalcon Singles - Oud for Falcon
Falcon Singles - Oud for Falcon 2016 Simon Stockhausen Installation As there is no default location for 3rd party sound libraries for Falcon, you can just install the folder Oud which you extracted from
More informationA Look at Un-Electronic Musical Instruments
A Look at Un-Electronic Musical Instruments A little later in the course we will be looking at the problem of how to construct an electrical model, or analog, of an acoustical musical instrument. To prepare
More informationDistortion products and the perceived pitch of harmonic complex tones
Distortion products and the perceived pitch of harmonic complex tones D. Pressnitzer and R.D. Patterson Centre for the Neural Basis of Hearing, Dept. of Physiology, Downing street, Cambridge CB2 3EG, U.K.
More informationTURN2ON BLACKPOLE STATION POLYPHONIC SYNTHESIZER MANUAL. version device by Turn2on Software
MANUAL version 1.2.1 device by Turn2on Software http://turn2on.ru Introduction Blackpole Station is a new software polyphonic synthesizer for Reason Propellerhead. Based on 68 waveforms in 3 oscillators
More informationOutline. Communications Engineering 1
Outline Introduction Signal, random variable, random process and spectra Analog modulation Analog to digital conversion Digital transmission through baseband channels Signal space representation Optimal
More informationNon-stationary Analysis/Synthesis using Spectrum Peak Shape Distortion, Phase and Reassignment
Non-stationary Analysis/Synthesis using Spectrum Peak Shape Distortion, Phase Reassignment Geoffroy Peeters, Xavier Rodet Ircam - Centre Georges-Pompidou, Analysis/Synthesis Team, 1, pl. Igor Stravinsky,
More informationTIME DOMAIN ATTACK AND RELEASE MODELING Applied to Spectral Domain Sound Synthesis
TIME DOMAIN ATTACK AND RELEASE MODELING Applied to Spectral Domain Sound Synthesis Cornelia Kreutzer, Jacqueline Walker Department of Electronic and Computer Engineering, University of Limerick, Limerick,
More informationDept. of Computer Science, University of Copenhagen Universitetsparken 1, DK-2100 Copenhagen Ø, Denmark
NORDIC ACOUSTICAL MEETING 12-14 JUNE 1996 HELSINKI Dept. of Computer Science, University of Copenhagen Universitetsparken 1, DK-2100 Copenhagen Ø, Denmark krist@diku.dk 1 INTRODUCTION Acoustical instruments
More informationJOURNAL OF OBJECT TECHNOLOGY
JOURNAL OF OBJECT TECHNOLOGY Online at http://www.jot.fm. Published by ETH Zurich, Chair of Software Engineering JOT, 2009 Vol. 9, No. 1, January-February 2010 The Discrete Fourier Transform, Part 5: Spectrogram
More informationSINOLA: A New Analysis/Synthesis Method using Spectrum Peak Shape Distortion, Phase and Reassigned Spectrum
SINOLA: A New Analysis/Synthesis Method using Spectrum Peak Shape Distortion, Phase Reassigned Spectrum Geoffroy Peeters, Xavier Rodet Ircam - Centre Georges-Pompidou Analysis/Synthesis Team, 1, pl. Igor
More informationREpeating Pattern Extraction Technique (REPET)
REpeating Pattern Extraction Technique (REPET) EECS 32: Machine Perception of Music & Audio Zafar RAFII, Spring 22 Repetition Repetition is a fundamental element in generating and perceiving structure
More informationHCS 7367 Speech Perception
HCS 7367 Speech Perception Dr. Peter Assmann Fall 212 Power spectrum model of masking Assumptions: Only frequencies within the passband of the auditory filter contribute to masking. Detection is based
More informationTranscription of Piano Music
Transcription of Piano Music Rudolf BRISUDA Slovak University of Technology in Bratislava Faculty of Informatics and Information Technologies Ilkovičova 2, 842 16 Bratislava, Slovakia xbrisuda@is.stuba.sk
More informationDeveloping a Versatile Audio Synthesizer TJHSST Senior Research Project Computer Systems Lab
Developing a Versatile Audio Synthesizer TJHSST Senior Research Project Computer Systems Lab 2009-2010 Victor Shepardson June 7, 2010 Abstract A software audio synthesizer is being implemented in C++,
More informationMusical Acoustics, C. Bertulani. Musical Acoustics. Lecture 13 Timbre / Tone quality I
1 Musical Acoustics Lecture 13 Timbre / Tone quality I Waves: review 2 distance x (m) At a given time t: y = A sin(2πx/λ) A -A time t (s) At a given position x: y = A sin(2πt/t) Perfect Tuning Fork: Pure
More informationADDITIVE SYNTHESIS BASED ON THE CONTINUOUS WAVELET TRANSFORM: A SINUSOIDAL PLUS TRANSIENT MODEL
ADDITIVE SYNTHESIS BASED ON THE CONTINUOUS WAVELET TRANSFORM: A SINUSOIDAL PLUS TRANSIENT MODEL José R. Beltrán and Fernando Beltrán Department of Electronic Engineering and Communications University of
More informationLaboratory Assignment 4. Fourier Sound Synthesis
Laboratory Assignment 4 Fourier Sound Synthesis PURPOSE This lab investigates how to use a computer to evaluate the Fourier series for periodic signals and to synthesize audio signals from Fourier series
More informationAcoustics and Fourier Transform Physics Advanced Physics Lab - Summer 2018 Don Heiman, Northeastern University, 1/12/2018
1 Acoustics and Fourier Transform Physics 3600 - Advanced Physics Lab - Summer 2018 Don Heiman, Northeastern University, 1/12/2018 I. INTRODUCTION Time is fundamental in our everyday life in the 4-dimensional
More informationPOLYPHONIC PITCH DETECTION BY MATCHING SPECTRAL AND AUTOCORRELATION PEAKS. Sebastian Kraft, Udo Zölzer
POLYPHONIC PITCH DETECTION BY MATCHING SPECTRAL AND AUTOCORRELATION PEAKS Sebastian Kraft, Udo Zölzer Department of Signal Processing and Communications Helmut-Schmidt-University, Hamburg, Germany sebastian.kraft@hsu-hh.de
More informationSGN Audio and Speech Processing
SGN 14006 Audio and Speech Processing Introduction 1 Course goals Introduction 2! Learn basics of audio signal processing Basic operations and their underlying ideas and principles Give basic skills although
More informationCommunications Theory and Engineering
Communications Theory and Engineering Master's Degree in Electronic Engineering Sapienza University of Rome A.A. 2018-2019 Speech and telephone speech Based on a voice production model Parametric representation
More informationInternational Journal of Modern Trends in Engineering and Research e-issn No.: , Date: 2-4 July, 2015
International Journal of Modern Trends in Engineering and Research www.ijmter.com e-issn No.:2349-9745, Date: 2-4 July, 2015 Analysis of Speech Signal Using Graphic User Interface Solly Joy 1, Savitha
More informationPlaits. Macro-oscillator
Plaits Macro-oscillator A B C D E F About Plaits Plaits is a digital voltage-controlled sound source capable of sixteen different synthesis techniques. Plaits reclaims the land between all the fragmented
More informationLinguistic Phonetics. Spectral Analysis
24.963 Linguistic Phonetics Spectral Analysis 4 4 Frequency (Hz) 1 Reading for next week: Liljencrants & Lindblom 1972. Assignment: Lip-rounding assignment, due 1/15. 2 Spectral analysis techniques There
More informationPattern Recognition. Part 6: Bandwidth Extension. Gerhard Schmidt
Pattern Recognition Part 6: Gerhard Schmidt Christian-Albrechts-Universität zu Kiel Faculty of Engineering Institute of Electrical and Information Engineering Digital Signal Processing and System Theory
More information