Onset Detection Revisited

Transcription

1 Austrian Research Institute for Artificial Intelligence Vienna, Austria 9th International Conference on Digital Audio Effects

2 Outline Background and Motivation 1 Background and Motivation Introduction Difficulties Previous Work 2 Spectral Flux Phase Deviation Complex Domain 3 Methodology and Data Onset Selection Results and Discussion

3 Musical Onset Detection Introduction Difficulties Previous Work Aim: detect the start of musical tones Motivations: content-based analysis and retrieval of music information, automatic transcription, etc. This study: verify and extend results of Bello et al. (TSAP 2005) propose new onset detection functions test on a large database of solo piano performances test set 100 times larger than Bello et al. s

4 Introduction Difficulties Previous Work Why is Onset Detection Difficult? Real-world data: complex polyphonic music Simultaneous or quasi-simultaneous notes Masking Chord asynchrony is at the limits of human perception (10 s of milliseconds) How many onsets in a chord? Difficult to evaluate methods quantitatively

5 Previous Work Background and Motivation Introduction Difficulties Previous Work Onset detection literature reviewed by Bello et al. (IEEE TSAP 2005) Empirical comparison of various methods using a standard data set Methods based on short-term spectral features most widely used most successful: winners of MIREX 2005 (see Downie, 2005) Further review by Collins (AES 118th Convention, 2005)

6 Spectral Flux Phase Deviation Complex Domain Peaks coincide with times of note onsets Low sampling rate w.r.t. audio Detect change in properties of audio signal Distinguish between various types of change: onsets, offsets, vibrato, amplitude modulation, noise Basic ideas: increase in energy in some frequency band(s) irregularity in phase derivative in some frequency bands(s) combinations of phase and amplitude/energy features

7 Audio Preprocessing Spectral Flux Phase Deviation Complex Domain Audio data: 44.1kHz mono STFT Hamming window Window length 2048 samples (46ms) Hop length 441 samples (10ms) Overlap 78.5%

8 Spectral Flux Phase Deviation Complex Domain Spectral Flux (SF): Existing Method Change in magnitude from frame to frame in each frequency bin Only positive changes Summed across frequency

9 Spectral Flux Phase Deviation Complex Domain Phase Deviation (PD): Existing Method Phase irregularities indicate transients Instantaneous frequency: first difference of phase Phase deviation: second difference of phase Averaged across frequency

10 Spectral Flux Phase Deviation Complex Domain Phase Deviation: Proposed Improvements Signal energy is concentrated in a few frequency bins Remaining components have low energy and random phase Proposal: weighted phase deviation (WPD) phase deviation is multiplied by magnitude and summed over frequency optionally, the WPD is normalised by dividing by the sum of magnitudes (NWPD) Joint consideration of magnitude and phase

11 Spectral Flux Phase Deviation Complex Domain Complex Domain (CD): Existing Method Joint consideration of magnitude and phase Search for departures from steady-state behaviour Steady-state: magnitude and instantaneous frequency remain constant Predict amplitude and phase based on 2 previous frames Sum absolute values of deviations (in complex plane) from predictions

12 Spectral Flux Phase Deviation Complex Domain Complex Domain: Proposed Improvements Complex domain method doesn t distinguish between increases and decreases in magnitude cf spectral flux, where only positive changes are considered Proposal: rectified complex domain function (RCD) Only sum differences where the magnitude is increasing

13 Methodology and Data Methodology and Data Onset Selection Results and Discussion Evaluation of multiple onsets (chords) Comparisons difficult (parameter settings for corresponding points on ROC curve) Precision (P), recall (R) and F-measure (F = 2PR P+R ) Ground truth data is hard to find Computer-monitored piano (e.g. Bösendorfer SE) is an exception Alternative is hand-labelled data 2 data sets: Hand-labelled short excerpts from various instruments (1060 onsets) from Bello et al. (2005) 4 hours of complex piano music ( onsets)

14 Onset Selection Background and Motivation Methodology and Data Onset Selection Results and Discussion Peak picking function is critical to onset detection performance Thresholds determine balance of false detections (false positives) and missed detections (false negatives) Optimal balance is application-specific Local maximum Peak higher than the local average (+ threshold) Peak not masked by previous higher peak (exponential decay)

15 Results (F-measure) Methodology and Data Onset Selection Results and Discussion PN data PP data NP data CM data Piano SF* SF PD* PD WPD NWPD CD RCD

16 Discussion Background and Motivation Methodology and Data Onset Selection Results and Discussion Large discrepancies with published results implementation details affect results radically PD* implemented with magnitude threshold SF vs SF* shows improved peak-picking WPD and NWPD are significant improvements over PD RCD and CD differences are test-set specific Complex piano music has slightly worse results than PP data SF, CD, NWPD have similar overall performance Accuracy: SF 8.8ms; CD 12.8ms; NWPD 10.3ms SF is simplest and has most precise onset times