MAKE SOMETHING THAT TALKS?

Transcription

1 MAKE SOMETHING THAT TALKS?

2 Modeling the Human Vocal Tract pitch, timing, and formant control signals pitch, timing, and formant control signals lips, teeth, and tongue formant cavity 2 formant cavity 1 pulse source resonant filter resonant filter noise, dynamics vocal folds

3 Sound Path Functional Blocks vowels: resonant filter pulse source envelope (attack, decay) consonants: resonant filter noise source resonant filter envelope (attack, decay) Combined outputs creates a functional approximation of the human vocal tract!

4 Sound Path Functional Blocks vowels: = resonant filter pulse source envelope (attack, decay) consonants: resonant filter noise source resonant filter envelope (attack, decay) Combined outputs creates a functional approximation of the human vocal tract!

5 Mixing the output of two filters approximates double-peak speech resonances. db Freq. ee ah ih oo Spectra captured with Visual Analyzer software by Alfredo Accattatis

6 What's in a Word? Phonemes! Grist = 1 word --- it has 1 syllable --- but 5(!) phonemes: /g/ /r/ /ih/ /s/ /t/ We will be making syllables by combining phonemes. If we want to synthesize grist, it will require that we synthesize 5 phonemes and link them together in time. But that's pretty difficult. Let's try to simplify. Our system will allow us to use 1- or 2-phoneme syllables, as long as they start with the consonant. And let's use simple consonants too. Things like koo, bah, or toe. Koo has 2 phonemes: /k/ and /oo/. Here's a timing diagram of it: /k/ /oo/ So we need to build a machine that first produces the consonant, then after that produces the vowel. This is something we can handle. So let's discuss how to build the control circuits of the synthesizer.

7 Spectral Control Your brain controls the resonant frequencies of formants by changing the shape of the formant cavities in your throat and mouth. The littlebits Synth Kit controls the resonant frequencies of voltage controlled filters by modulating the control voltage to the filter. The microsequencer has 4 independent steps, each having its own knob to set the control voltage for that step V V db Freq.

8 Timing Control This circuit uses a sawtooth (ramp) wave from oscillator and a few logic modules to generate a normal positive-edge-triggered clock, and a delayed clock. The length of the delay varies depending on the frequency of the oscillator, so you will need to experiment with different frequencies until the delay sounds right. Positive edge triggered clock Delayed clock

9 Control Path Functional Blocks timing: spectra: 5V to consonant formant Positive edge triggered clock 0V Variable control voltages ramp source 5V 0V to vowel formant 2 Variable control voltages Functional approximation of the brain and nerve connections to the vocal tract! delayed clock 5V 0V to vowel formant 1 Variable control voltages

10 Control Path Functional Blocks timing: spectra: 5V to consonant formant 0V Variable control voltages Positive edge triggered clock ramp source == 5V 0V Variable control voltages to vowel formant 2 Functional approximation of the brain and nerve connections to the vocal tract! delayed clock 5V 0V to vowel formant 1 Variable control voltages

11 Building the Synth Tune All Three Filters: 1. Set filter cutoff to ~ 45% 2. Set filter peak to 100% 3. Set speaker volume to 20% 4. Connect the circuit below, and listen to the pitch produced. 45% 100% 20% Indicator line 5. Now listen to each filter in succession, making small adjustments with the cutoff knob until all three produce the same pitch. 6. Now don't change the cutoff of the filters ever again, unless you want to re-tune them. 7. Turn the peak control down (counterclockwise) until the filter just stops oscillating (until it stops producing a tone).

12 Building the Synth Build and test the Clock Generator: 1. Set oscillator mode to saw 2. Oscillator tune knob does not matter 3. Set oscillator pitch to about 10% 4. Connect the circuit below: fork inverter LED o1 led = Consonant clock wire power dimmer oscillator NOR 5. Turn on the power and observe the LEDs 6. Adjust the dimmer and oscillator knobs to set the LEDs to slow flashing. 7. The delayed clock LED should stay mostly on, and turn off then back on quickly 8. The delayed vowel clock LED should turn on AFTER the consonant clock LED. XOR LED o1 led Vowel clock (delayed)

13 Building the Synth Build and test the Consonant Generator: 1. Set microsequencer mode to step 2. Set all knobs on microsequencer to approximately 50% 3. Set random mode to noise 4. Set speaker volume to approximately 50% 5. Set dimmer to 0% (off) 6. Build the circuit pictured. 7. Turn the dimmer on slowly to advance the sequencer 8. Check that at each step you can move the knobs on the sequencers to change the sound changes 9. Remove the power, dimmer, and speaker modules when done. speaker envelope filter random power dimmer micro sequencer

14 Building the Synth Build and test the Vowel Generator: 1. Set microsequencer mode to step on all microsequencers 2. Set all knobs on all microsequencers to approximately 50% 3. Set set number mode to values 4. Set oscillator mode to saw 5. Set oscillator pitch to 30% 6. Oscillator tune knob does not matter 7. Set speaker volume to approximately 50% 8. Set dimmer to 0% (off) 9. Set envelope attack to 0%, and decay to max 10. Build the circuit pictured 11. Turn the dimmer on slowly to advance the sequencers 12. Check that at each step you can move the knobs on the sequencers to change the values on the number modules and that the sound changes 13. Remove the power, dimmer, and speaker modules when done. micro sequencer number filter speaker envelope power dimmer oscillator filter fork micro sequencer number

15 The Complete Vocal Synth Assemble the Components: 1. Connect the consonant generator to the clock generator 2. Connect the vowel generator the to clock generator 3. Connect a module to the outputs of the vowel and clock generators 4. Connect the module to the speaker 5. Switch on power and turn up the dimmer 6. Verify that the micro sequencers step and are synchronized 7. Set knobs to max 8. Verify that you hear vowels and consonants.

16 Step 4 Step 3 Step 2 Step 1 Tuning the Synth 1. Set vowel knobs to max 2. Use dimmer to cycle to step 1 3. Use micro sequencer knobs to set filter values 4. Cycle to next step and set filter values 5. Repeat for up to 4 steps 6. Start continuous cycling with dimmer 7. Adjust envelope and settings for best sound. You will probably need to reduce the consonant volume. Often, increasing the vowel attack can improve quality. 8. Keep in mind that these settings are starting points! You will need to make Very Small Adjustments as you listen to the sounds. With time and practice, you will learn how to quickly achieve good results. Cons onant envel ope Vowelconsonant Vowel envel ope Dimmer HiFilt LoFilt

17 Step 4 Step 3 Step 2 Step 1 Word 1: Cookie Vowel envelope: Attack: ~20% Decay: max Consonant envelope: Attack: ~5% Decay: ~10% Cons onant envel ope Vowelconsonant Sequencer Settings: Step phoneme LoFilt HiFilt Cons "koo" "kee" Vowel envel ope Dimmer HiFilt LoFilt

18 Step 4 Step 3 Step 2 Step 1 Word 2: Barbecue Vowel envelope: Attack: ~20% Decay: max Consonant envelope: Attack: ~5% Decay: ~10% Cons onant envel ope Vowelconsonant Sequencer Settings: Step phoneme LoFilt HiFilt Cons "bah" "bee" "koo" Vowel envel ope Dimmer HiFilt LoFilt

19 A Few Enhancements Step 1 Step 2 Step 3 Step 4 Adding delays in front of the filters can add motion to the vowels, which can be more life-like. Start with delay at min, and turn up the feedback. Then, adding a dimmer in front of the vocal oscillator allows you to add inflection. Move the vocal dimmer in small amounts during speech to give expressive pitch changes to the word.

20 Step 4 Step 3 Step 2 Step 1 Word 3: Autobahn Vowel envelope: Attack: ~20% Decay: max Consonant envelope: Attack: ~5% Decay: ~10% Cons onant envel ope Vowelconsonant Sequencer Settings: Step phoneme LoFilt HiFilt Cons "ah" "toh" "bah" "n" Vowel envel ope Dimmer HiFilt Delay LoFilt Delay Vocal Dimmer

21 Step 4 Step 3 Step 2 Step 1 Word 4: Robot Vowel envelope: Attack: ~20% Decay: max Consonant envelope: Attack: ~5% Decay: ~10% Sequencer Settings: Step phoneme LoFilt HiFilt Cons "roh" "bah" "t"

22 Control Reference All filters tuned to ~800Hz (about 45%) peak set above 50% Vowel envelope: Attack: ~20% Decay: max Consonant envelope: Attack: ~0%, varies Decay: ~10%, varies Vowel Formant Frequencies: phoneme f1 f2 filt#1 filt# "oh" "ah" "ee" "oo" "ih" "eh" "uh" "er" "ll" Consonant Formant Frequencies: Cons. Seq. cons. f1 Knob Setting w y r l f v s Z ch jh p b t d k g m n Adapted from Dennis H. Klatt p987

23 Further Research Reference Vocal Synthesis History: Voder Vocoder Speak-N-Spell Speech Science: Homer Dudley (Voder, Vocoder) Dennis H. Klatt (Rules based synthesis) *good bibliography Good search Terms: vowel formants consonant formants speech synthesis rules based speech synthesis formant synthesis

24 Next Steps 1: The consonant block should really have two filters, just like the vowel block, but using noise instead of an oscillator for the source. i33 envelope i32 filter o24 synth speaker o21 number w7 fork i36 microsequencer i34 random i36 microsequencer i33 envelope w7 fork w10 inverter p1 power i6 dimmer i31 oscillator w1 wire i32 filter o21 number w15 NOR w7 i36 microsequencer fork w17 i6 dimmer i31 oscillator XOR i32 filter o21 number w1 wire w1 wire o21 number i32 filter w1 wire i36 microsequencer

25 Next Steps 2: Control and timing is difficult. Using a programmable controller would improve intelligibility. If we could program each syllable or word individually, all the detailed timings and filter movements would make speech more realistic. Our new Arduino module, for example, would be a perfect fit for this job. i33 envelope o21 number o24 synth speaker w7 fork i36 microsequencer w1 wire i36 microsequencer o21 number w7 fork w10 inverter w1 wire i33 envelope p1 power i6 dimmer i31 oscillator w1 wire i32 filter w17 XOR w15 NOR w7 fork i6 dimmer i36 microsequencer littlebits Module o21 number w1 wire i36 microsequencer o21 number i32 filter i32 filter i34 random Arduino Cock i32 filter CV1 CV2 Power i31 oscillator

26 Next Steps 2: Here you see how you could use two littlebits Arduino modules to replace about 17 regular modules and get improved functionality.. i33 envelope littlebits Arduino Module i32 filter o24 synth speaker Clock CV1 CV2 Power i32 filter i33 envelope i34 random i32 filter littlebits Arduino Module Clock CV1 i32 filter CV2 Power i31 oscillator