Performance variability enables adaptive plasticity ‘crystallized’ adult song

2011.02.22Performance variability enablesadaptive plasticity ‘crystallized’ adult song Tumer and Brainard, Nature 2007 Introduction by K. Sasahara

Introduction• Subtle variation in performance is simply a ‘noise’ that the nervous system is unable to control or that remains below threshold for behavioral relevance?• Such variation enables trial-and-error learning, in which the motor system generates variation and differentially retains behaviors that produce better outcomes?• This paper experimentally tested the second possibility.

Prerequisite knowledge• Birdsong is a learned vocal behavior that requires precise motor control and auditory feedback• Behavior contingency Antecedent → Behavior → Consequence (reinforcer) Song control circuit Developmental learning of song (from http://people.eku.edu/ritchisong/avian_biology.htm)

ARTICLES a Methods and Materials c NATURE | Vol 450 | 20/27 December 2007 Song Shifted output feedback Figure 1 Technique for manipulating auditory feedback. (a) Crystalli from an adult Bengalese finch. Spectrographic representation shows 8 A B C power at each frequency (color scale) as a function of time. Three ha wit ‘Crystallized’ adult songFrequency (kHz) A A features are labeled A, B and C. (b) Each bird was fit with a set of dat a headphonesathat housedca pair of speakers. A microphone in the dir b d e e cag inset) provided input to online sound-processing hardware, whichdem wa manipulate the pitch of song. Processed acoustic signals were thenvar +1σ the headphone speakers via a flexible cable (not shown in photograp 1 Adult Bengalese finch 100 ms played through the speakers. (c) An upward (+100 cents) shift indet th C b B B auditory feedback was introduced by the headphone system. For eac fre harmonic features labeled in a, the left spectrogram shows the bird’s (Fi Mean F0 output and the black triangle shows the frequency of the harmonic f to The right spectrogram shows the pitch-shifted auditory feedback pla and through the headphones and the red triangle shows the frequency of (Fi Pitch shift harmonic feature in the shifted song. Black triangles are repeated ne –1σ the spectrograms on the right for comparison. 5 kHz Microphone C day C (Fi 100 ms syllable, we quantified changes in pitch by measuring ch each T Head b Escape Hit Aft either the*fundamental frequency or the frequency of one of th * we Speakers harmonics (a harmonic feature, see Online Methods). In tow a re (from Sober and Brainard, Nature Neursci. 2009) toRESULTS a 10 ally 5 kHzA set of lightweight headphones was custom-fit to each bird in the D A Freq (kHz)study to generate online shifts in the pitch of auditory feedback (an B C 5 aexample of crystallized song from one bird in our study is shown in 250 ms Trigger timeFig. 1a). A microphone in each bird’s cage relayed acoustic signals c Baseline White noise Baseline ethrough sound-processing hardware capable of generating arbitrary Binary signal (negativeDownward Upward reinforcer) Fundamentalshifts in pitch. These pitch-shifted acoustic signals were then played 100 ms escape escape Escape Hitback through speakers in the headphones (Fig. 1b) with an average b Baseline Natural Shift 0.4 feedback Recoveryprocessing delay of B7 ms. Shifts in the pitch of auditory feedback and 0.2 100

b Escape Hit After at least 3 days of reinforcement, contingent w * * were terminated. In every case, the fundamental fre Differential reinforcement can adaptively towards its original range (Fig. 2c). Hence, the nervo a representation of the initial song and both the capa to return song towards its original structure in the ab alter features of adult song ally imposed drive. 5 kHz a 250 ms White noise on n=1 n=7 Trigger time 2.4Before c e frequency (kHz) Baseline Baseline Fundamental Downward Upward Escape Hit escape escape Even ‘crystallized’ adult song 2.3 0.4 0.2 has 2.2 small natural variations Probability -2 -1 0 1 2 3 F0 = 2281± 27 Hz Probability 0.1 0.2 Appropriate contingency on (days) Time since feedback Variation: ~ 1% directs either increases or b c 0 decreases in pitch White noise on 0 2.2 2.3 –5 –2.5 0 2.5 5 3 3 Fundamental frequency (kHz) Fundamental frequency (z-score) Large reduction in hit rate fundamental frequency (z-score) fundamental frequency (z-score)After d Day 3 f Downward Day 3 Upward Adaptive shift in mean Adaptive shift in mean escape escape 2 2 Escape Hit 0.4 0.2 1 1 Hit rate: 91% → 17 % Probability Probability 0.2 0.1 0 0 –1 –1 0 0 Last day feedback on Morning Baseline Evening Day 3 Last day 2.2 2.3 –5 –2.5 0 2.5 5 Fundamental frequency (kHz) Fundamental frequency (z-score) Figure 1 | Differential reinforcement can adaptively alter features of { Day 1

ally imposed drive.and for all birds significant changes occurred within the first day 5 kHz(Fig. 2b, ‘evening’). The median number of syllables sung withinthe first half-day and full day were 605 and 1,179, respectively. By a(Fig. 2b). c 250 ms e Adaptive shifts in fundamental frequencyday 3, fundamental frequency stabilized at nearly Trigger time values asymptotic 2.4 White noise on frequency (kHz) The inducedBaseline in fundamental frequencyBaseline rapidly. occur rapidly and recover Fundamental changes Downward recovered Upward escape white escape burstsAfter at least 3 days of reinforcement, contingent noise 2.3 Escape Hitwere terminated. In every case, the fundamental frequency reverted 0.4towards its original range (Fig. 2c). Hence, the nervous system retains 0.2 2.2a representation of the initial song and both the capacity and impetusto return song towards its original structure in the absence of extern- Probability -2 -1 0 1 2 3 4 5 Probabilityally imposed drive. 0.1 0.2 n=1 n=7 Time since feedback on (days) b c a White noise on White White noise on noise off 0 0 2.4frequency (kHz) 2.2 2.3 –5 –2.5 0 2.5 5 3 3 Fundamental Fundamental frequency (kHz) Fundamental frequency (z-score) 2.3 fundamental frequency (z-score) fundamental frequency (z-score) d Day 3 f Day 3 Downward Upward Adaptive shift in mean Adaptive shift in mean 2.2 escape escape 2 2 Escape Hit 0.4 -2 -1 0 1 2 3 4 5 0.2 1 1 Time since feedback on (days) b c Probability Probability 0.2 0.1 White 0 0 F0 approached the asymptotic range White noise on noise off within one day (in a, 7hrs3= 600songs) 3 –1 –1 0 0ndamental frequency (z-score) Day 3 feedback Last day feedback on off ndamental frequency (z-score) Morning Baseline Evening Day 3 Last day 2.2 2.3 –5 –2.5 0 2.5 5 Nervous system retains the initial song Fundamental frequency (kHz) Fundamental frequency (z-score) Adaptive shift in mean Adaptive shift in mean 2 2 Figure 1 | Differential reinforcement can adaptively alter features of { representation b, c, d, e,the capacity to with little adult song. a, Syllables (a, and e) are normally produced Day 1 feedback variation. Three songs are shown for which the fundamental frequency of ‘a’ returnstandard deviations of the baseline 1distribution (Supplementary spanned 2 to it 1 on Recording 1 contains corresponding audio files). b, White noise bursts Figure 2 | Adaptive shifts in fundamental frequency occur rapidly and (‘hits’) were targeted at higher pitched versions of ‘a’. c, Baseline recover. a, Fundamental frequency of targeted syllables for one adult bird fundamental frequency distribution for ‘a’, showing overall mean (triangle) 0 0 (age 334 days). Fundamental frequency progressively increased during the

system was able to detect and respond precisely to that contingency. This specificity indicates an impressive capacity of the nervous b 4 Mean shift in fundamentalsystem to modify discrete features of song independently. This is Changes are restricted to targeted frequency (z-score)appropriate for vocal learning, where birds match their song to 2 Short delayrapidly varying features of an acoustic model. If modification of b Targetone song feature generalized to cause modification of others, learning 0 syllableprogress26. features of songmight still proceed, but such interference would probably slow its In theory, reinforcement signals can drive learning even at long –2 –4 dlatencies to the actions that precipitate them15. For complex motor –400 –300 –200 –100 0 100 200 300 400 Mean time from target syllable (ms) 5 kHz a Target c 10 3 Target 8 100 ms Frequency (kHz) +100 ms delay Control Mean shift in feature 6 2 c Feedback off Feedback on 4 (z-score) 1 Short delay +100 ms delay Short delay Fundamental frequency (kHz) 2 2.4 Downward Downward 0 escape escape –400 –300 –200 –100 0 100 200 300 400 0 Mean time from target syllable (ms) 2.3 b –1 Upward 4 Mean shift in fundamental Fundamental Duration Volume escape Entropy 2.2 frequency (z-score) 2 frequency 0 3 7 15 21 25 28 Time (days) Figure 3 | Changes are restricted to targeted features of song. a, Spectrogram Fig 0 illustrating analysed features for target (red bar) and control (blue bars) a, b d Short +100 ms Short +100 ms syllables of an individual experiment. b, Mean changes in fundamental (b) –2 –4 Nervous system can: 3 delay (red) delaycontrol (blue) syllables. Squares represent two frequency for target and 1 delay delay experiments for song illustrated in a. Data from 3 additional birds (circles, dev shi –400 –300 –200 –100 0 100 200 300 400 • detect and respond to the imposed triangles, diamonds) are shown without corresponding spectrograms. Filled and open symbols indicate experiments with upward and downward shifts in del usi contingency (pitch-white noise) ndamental frequency (z-score) Mean time from target syllable (ms) 0.8 fundamental frequency, respectively. c, Spectral characteristics other than 2 dev c Adaptive shift in mean 3 Target • modify song features0.6independently fundamental frequency were not altered for either target (red) or control (blue) syllables. Bars indicate mean 6 standard deviation. Sym rein Control 1242 n shift in feature Hit rate 2 1 ©2007 Nature Publish 0.4 (z-score) 1 0 0.2

In theory, reinforcement signals can drive learning even at long latencies to the actions that precipitate them15.December 2007 motor NATURE | Vol 450 | 20/27 For complex 5 kHz Delayed feedback prevents adaptive pitch shiftsaltering a Target skills, however, the nervous system might detect the contingency f larger 10 more effectively at shorter delays. We tested the importance of delaychanges by varying8 time between measurement of fundamental frequency the 100 ms Frequency (kHz) argeted +100 ms delay 6en they a Detect time c Feedback off Feedback on g. 3a, b, 4 Target syllable Trigger time Short delay +100 ms delay Short delay o fun- Fundamental frequency (kHz) 2 s dura- 2.4 0 Downward Downwardugh the escape escapend feed- –400 –300 –200 –100 0 100 200 300 400nervous Mean time from target syllable (ms) 2.3ngency. bnervous Upward 4 5 kHz Mean shift in fundamental This is escape 2.2 frequency (z-score) ong to 2 Short delay 28 0 3 7 15 21 25 tion of b Target Time (days)earning 0 syllableslow its d Short +100 ms Short +100 ms –2 delay delay delay delay at long –4 3 1 motor –400 –300 –200 –100 0 100 200 300 400 fundamental frequency (z-score) Mean time from target syllable (ms) 0.8 2 c Adaptive shift in mean 3 100 ms Target +100 ms delay 0.6 Control FundamentalMean shift in feature Hit rate c 2 Feedback off Feedback on 1 → adaptive pitch shifts delay Short delay Short delay +100 ms delay Short 0.4 frequency (kHz) (z-score) 2.4 Long delay (+100ms) → no systematic changes 400 1 Downward escape Downward escape 0 0.2 2.3 0 Normally –1 predictable timing between Upward escape –1 0 Baseline Day 3 Day 5 Baseline Day 3 Day 5 Day 8 Day 12 Baseline Day 3 Day 5 Baseline Day 3 Day 5 Day 8 Day 12 premotor0 activity andTime (days) Volumeconsequence sensory21 25Entropy 2.2 3 Fundamental Duration15 7 28 (< 70ms)| Changes are+100 ms to targeted features of song.ms, Spectrogram d Figure 3 Short frequency restricted Short +100 a Figure 4 | Delayed feedback prevents adaptive pitch shifts. delay delay delay delay a, b, Spectrograms illustrating short delay (a) and 1100 ms delay illustrating analysed features for target (red bar) and control (blue bars)

LETTERS ined Incremental adjustment of threshold internally by an efference copy of premotor activity17. Consistent with this possibility, studies in both birds and mammals indicate that se er ’). drives large pitch changes some movement variability derives from central neural activity rather than the periphery7,8,29,30, suggesting that variability may be actively generated for motor exploration. Regardless of mechanism, our p- data indicate that natural variations present in crystallized adult songes, are not simply noise but rather can be exploited for trial-and-error ged). aes. Fundamental frequency (kHz)tic 2.6ite ry 2.5edble 2.4 White noisecal 2.3nges; –2 0 2 4 6 8 10 12 nt Time since feedback onset (days) us ben 0.2 Baseline Day 13 n-lly ofon Probability 0.1ult (Video from supplemental materials) 2.5 kHz se25 .on 100 mshe 0nt 2.3 2.4 2.5 2.6 2.7 in Fundamental frequency (kHz)ig. re c Baselineel- 0.3

0.1ult Prob 2.5 kHz se 25 . (Continued)on 100 mshe 0nt 2.3 2.4 2.5 2.6 2.7 in Fundamental frequency (kHz)ig. re c Baselineel- 0.3 e- ld Probability 0.25). Dramatic remodeling oftal 0.1 vocalizations by an incrementalge ry differential reinforcement 0ng –15 –10 –5 0 5 10 15 e-b- Fundamental frequency (z-score) Current performance is surroundedme d Last day by a ‘halo’ of variation that enablesnto- trial-and-error exploration for 0.3 o’ continuous adaptive modiﬁcation Probability 0.2alsto 0.1 a-an 0 se –15 –10 –5 0 5 10 15 in Fundamental frequency (z-score)d-ng Figure 5 | Incremental adjustment of threshold drives large pitch changes.18 . a, Changes to fundamental frequency for one experiment. Points indicaten- mean (6 standard deviation) on each day. Shading indicates threshold for

Summary (from Graphon, Nature Neursci. 2008)• Reinforcement using binary feedback can direct precise, adaptive changes to crystallized adult song• Natural variations present in crystallized adult song are not simply noise but rather can be exploited for trial-and-error learning

Performance variability enables adaptive plasticity ‘crystallized’ adult song

by soramame0518 on Feb 22, 2011

Accessibility

Tags

Upload Details

Usage Rights

2 Embeds 85

Statistics

Performance variability enables adaptive plasticity ‘crystallized’ adult song — Presentation Transcript

http://twilightcoffee.blogspot.com	76
http://twilightcoffee.blogspot.jp	9