Vol 466 | 22 July 2010 | doi:10.1038/nature09263
ARTICLES
Start/stop signals emerge in nigrostriatal
circuits during sequence learning
Xin Jin1 & Rui M. Costa1,2
Learning new action sequences subserves a plethora of different abilities such as escaping a predator, playing the piano, or
producing fluent speech. Proper initiation and termination of each action sequence is critical for the organization of
behaviour, and is compromised in nigrostriatal disorders like Parkinson’s and Huntington’s diseases. Using a self-paced
operant task in which mice learn to perform a particular sequence of actions to obtain an outcome, we found neural activity in
nigrostriatal circuits specifically signalling the initiation or the termination of each action sequence. This start/stop activity
emerged during sequence learning, was specific for particular actions, and did not reflect interval timing, movement speed or
action value. Furthermore, genetically altering the function of striatal circuits disrupted the development of start/stop
activity and selectively impaired sequence learning. These results have important implications for understanding the
functional organization of actions and the sequence initiation and termination impairments observed in basal ganglia
disorders.
Animal behaviour can be organized as sequences of particular actions
or movements1,2. The organization of behaviour as sequences of
actions is complex and requires the precise timing and ordering of
movements within a sequence. It also requires the proper initiation
and termination of the sequence; that is, identifying the first and the
last elements within the behavioural sequence. Although the study of
innate behavioural sequences and fixed action patterns controlled by
central pattern generators has received substantial attention3,4, the
neural mechanisms underlying the learning and execution of
acquired behavioural sequences are still largely unknown. The dorsal
striatum and its dopaminergic afferents have been implicated in skill
learning5,6 and action ‘chunking’7–9. Importantly, the initiation and
termination of sequences of voluntary movements is impaired in
disorders affecting the striatum and its dopaminergic inputs, such
as Parkinson’s10–12 and Huntington’s diseases11,13. Consistently, the
learning of new sequences is also compromised in disorders affecting
these circuits14–16. Furthermore, neuronal activity in the prefrontal
cortex, which projects to the striatum, can change during the signalled initiation and termination of a sequence of saccades17.
Although previous studies have reported changes in neural activity
in the striatum and the substantia nigra pars reticulata (SNr) during
the initiation of natural movement sequences18,19, the role of the
striatum and of nigrostriatal dopamine in the initiation and termination of newly acquired, self-generated action sequences has not been
explored. Here we show that as mice learn to perform a particular
behavioural sequence, neural activity specifically signalling the selfpaced initiation or termination of the newly acquired sequence
emerges in nigrostriatal circuits. In accordance, genetically manipulating the function of these circuits disrupts the development of
neural activity signalling sequence initiation or termination and
affects sequence learning.
Mice learn a specific sequence of actions
A group of mice (n 5 14) were trained in a self-paced operant task in
which a fixed number of lever presses (fixed-ratio-eight (FR8) schedule)
would earn a sucrose solution reward with no explicit stimuli signalling
1
the correct sequence length or reward availability (see Methods). The
average lever press rate increased with training (Supplementary Fig. 1;
P , 0.01), while the behaviour of the mice became more robustly
organized as discrete sequences of about eight presses (Fig. 1a, b). The
average number of presses per sequence increased significantly from
5.3 6 0.3 presses on day 1 to 7.6 6 0.3 and 8.1 6 0.3 on days 6 and 12,
respectively (P , 0.01; Fig. 1c); and after training it was no longer different from eight (day 1, P , 0.01; days 6 and 12, P . 0.05). Concomitantly, the average temporal duration of a sequence decreased
during the first six days of training (P , 0.01), and subsequently reached
a steady level (Fig. 1d). The inter-sequence interval (ISI) also decreased
with training (P , 0.01; Fig. 1e), whereas the within-sequence press rate
increased (P , 0.05; Fig. 1f). Importantly, the variability (measured by
coefficient of variation) of sequence length, sequence duration, ISI and
within-sequence press rate all decreased after six days of training
(P , 0.01 for all; Fig. 1g–j). It is unlikely that mice used explicit sensory
cues to determine when a reward would be available because often the
reward was delivered in the middle of a lever-press sequence and mice
continued to press before checking the magazine and consuming the
reward (Fig. 1b, inset), indicating that the initiation and termination of
a sequence of presses were largely self-determined. These results show
that a robust action sequence structure emerged with training.
Start/stop activity in nigrostriatal circuits
We recorded the neural activity in the dorsal striatum and substantia
nigra (dopaminergic neurons in compacta (SN DA) and GABAergic
neurons in reticulata (SN GABA)) during the emergence of action
sequences (Methods, Supplementary Fig. 2 and Supplementary
Table 1). Putative neuronal cell types were classified based on spike
waveform, firing properties and pharmacology; classification was
further confirmed using genetic20,21 and optogenetic tools22,23 (Supplementary Figs 3–6, 16 and 17). We found that striatal medium spiny
neurons (MSNs) and SN GABA neurons showed either a phasic
increase in firing frequency before lever pressing (Fig. 2a, d) or a
decrease of firing during lever pressing (Fig. 2b, c). Interestingly, SN
DA neurons usually showed a phasic increase but never a decrease in
Laboratory for Integrative Neuroscience, National Institute on Alcohol Abuse and Alcoholism, National Institutes of Health, 5625 Fishers Lane, Bethesda, Maryland 20892-9412, USA.
Champalimaud Neuroscience Programme at Instituto Gulbenkian de Ciência, Rua da Quinta Grande, Oeiras 2780-156, Portugal.
2
457
©2010 Macmillan Publishers Limited. All rights reserved
ARTICLES
NATURE | Vol 466 | 22 July 2010
1,400
1,450
Time (s)
80
1,500
4
0
0
0
,00 1,00
–
0
,00
–2
c
500
No. of lever press
120
1,000
Time (s)
1,500
0
0
0
0
0
00 ,000
00
00 ,000 ,000
,00 2,00 1,00
3
1,0 2,0
3
2
1,0
–
–
–3
Time (ms)
Time (ms)
0
d
30
80
15
0
0
0
,00
40
,00
–2
–3
0
,00
–1
e
0
200
220
Time (s)
80
6
2,000
Firing rate (Hz)
0
b 160
30
15
0
0
0
0
0
00 ,000 ,000
00 ,000 ,000
,00 2,00 1,00
3
1,0
2
3
2
–3
–
–
Time (ms)
0
1,0
Time (ms)
f
240
80
6
4
2
0
1
6
12
Days of training
h
0.6
0.4
0.2
0
1
6
12
Days of training
80
15
60
10
5
0
40
20
0
1
6
12
Days of training
i
j
1.2
1.2
1.0
1.0
0.8
0.8
0.6
0.4
0.6
0.4
0.2
0.2
0
0
1
6
12
Days of training
1
6
12
Days of training
1
6
12
Days of training
600
Within-sequence press rate
(presses/min)
8
500
f
e
20
ISI (CV)
d
10
300
400
Time (s)
Within-sequence press rate (CV)
200
ISI (s)
100
Duration of sequence (s)
0
Duration of sequence (CV)
Length of sequence (CV)
Length of sequence (presses)
0
700
80
60
40
20
0
1
6
12
Days of training
1.2
1.0
0.8
3
0
0
,00
–3
0
,00
–2
0
,00
–1
0
00 ,000 ,000
3
2
1,0
Time (ms)
40
0
80
80
Percentage
40
6
Percentage
Firing rate (Hz)
No. of lever press
12
–3
0
g
8
1
6
12
Days of
training
Percentage
0
1,350
Firing rate (Hz)
40
40
c
b
Firing rate (Hz)
No. of lever press
120
a
80
Firing rate (Hz)
No. of lever press
a 160
40
0
1
6
12
Days of
training
40
0
1
6
12
Days of
training
Figure 2 | Lever-press-related activity in nigrostriatal circuits.
a, b, Examples of MSNs showing positive (a) and negative (b) modulation of
firing rate in relation to lever pressing. c, d, Examples of negative (c) and
positive (d) modulation of firing rate of SN GABA neurons relative to lever
pressing. e, Example of positive modulation of firing rate in an SN DA
neuron before lever pressing. Top, raster plot where each dot indicates a
spike; bottom, perievent time histogram (PETH) of spiking activity for the
same neuron; time zero is the time of lever pressing (a–e). f, Percentage of
MSNs (left), SN GABA (middle) and SN DA (right) neurons displaying
press-related activity throughout learning. Error bars denote s.e.m.
0.6
0.4
0.2
0
1
6
12
Days of training
Figure 1 | Mice learn to perform a specific sequence of actions.
a, b, Example of the microstructure of the behaviour of a mouse during a
session on the first day of FR8 training (a) and the last day of FR8 training
(b). Each dot indicates a lever press, with red and black indicating the first
and final press of each individual sequence. The black and red solid lines on
the x axis represent magazine entries and licks, respectively. The black
dashed lines indicated the time of reward delivery and the corresponding
lever press. c–f, Average number of lever presses per sequence (c), sequence
duration (d), ISI (e) and within-sequence press rate (f), during the first, sixth
and twelfth day of FR8 training. g–j, Variability, measured as coefficient of
variance (CV), of sequence length (g), sequence duration (h), ISI (i) and
within-sequence press rate (j) during the first, sixth and twelfth day of FR8
training. Error bars denote s.e.m.
firing activity before lever pressing (Fig. 2e and Supplementary Fig. 7).
More than one third of all recorded MSNs and more than half of SN
GABA and SN DA neurons showed lever-press-related activity (Fig. 2f,
see statistics in Supplementary Information). However, there was no
significant change in the proportion of neurons showing lever-pressrelated activity as training progressed (P . 0.05 for all), indicating that
the changes observed in sequence behaviour (Fig. 1) were not caused by
overall changes in lever-press-related activity of these cell types.
Interestingly, when we investigated activity changes relative to
each of the different lever presses within a sequence, we found that
many MSNs showed a phasic increase in firing rate preceding the first
press, which was significantly higher than the rate increase preceding
the other presses within a sequence (P , 0.01; Fig. 3a, same neuron as
Fig. 2a). Similar sequence-initiation-specific activity was found in SN
GABA neurons (P , 0.01; Fig. 3b, same neuron as Fig. 2d) and SN
DA neurons (P , 0.01; Fig. 3c, same neuron as Fig. 2e). Moreover, we
also found neurons in all three areas that significantly modulated
their activity selectively before the final press of a sequence
(P , 0.001; Fig. 3d). Few neurons signalled both sequence initiation
and termination (P , 0.01; Fig. 3e) compared to those signalling only
initiation or termination (P , 0.01 for all cell types; Fig. 3i–k).
Importantly, the proportion of neurons showing start/stop-related
neural activity was much higher than that of neurons with activity
selectively related to the middle presses within a sequence (day 12,
P , 0.01 for all three areas), which was very low (Supplementary
Fig. 8).
Start/stop activity emerges with learning
Surprisingly, although there was no significant change in the proportion of lever-press-related neurons during training (Fig. 2), the percentage of sequence start/stop-related activity in striatal MSNs
increased from 16.0 6 2.5%, on day 1, to 28.4 6 3.2% and
27.8 6 2.9% on days 6 and 12, respectively (P , 0.01; Fig. 3f and
Supplementary Fig. 9). This increase was mainly observed from
day 1 to day 6 (P , 0.01), consistent with the time course of sequence
learning (Fig. 1c–f). Furthermore, this increase in start/stop activity
458
©2010 Macmillan Publishers Limited. All rights reserved
ARTICLES
NATURE | Vol 466 | 22 July 2010
was more prominent in sensorimotor striatum than in associative
striatum (Supplementary Fig. 10). A similar learning-related increase
for sequence start/stop-related activity was observed in SN GABA
neurons (20.6 6 2.7%, 40.6 6 2.5% and 39.4 6 3.3% for days 1, 6
and 12 respectively; P , 0.01; Fig. 3g) and SN DA neurons (27.1%,
36.0% and 44.4% for days 1, 6 and 12 respectively; x2 test, P , 0.05)
(Fig. 3h).
a
Firing rate (Hz)
1st
2nd
3rd
3rd to final 2nd to final
Final
20
20
20
20
20
20
10
10
10
10
10
10
0
–2,000
0
0
2,000 –2,000
0
0
2,000 –2,000
0
0
2,000 –2,000
Time (ms)
0
0
2,000 –2,000
0
0
2,000 –2,000
0
2,000
b
Firing rate (Hz)
1st
2nd
3rd
3rd to final 2nd to final
Final
28
28
28
28
28
28
14
14
14
14
14
14
0
–2,000
0
0
2,000 –2,000
0
0
2,000 –2,000
0
0
2,000 –2,000
Time (ms)
0
0
2,000 –2,000
0
0
2,000 –2,000
0
2,000
c
1st
2nd
3rd
3rd to final 2nd to final
Final
12
12
12
12
12
6
6
6
6
6
6
Firing rate (Hz)
12
0
–2,000
0
0
2,000 –2,000
0
0
2,000 –2,000
0
0
2,000 –2,000
Time (ms)
0
0
2,000 –2,000
0
0
2,000 –2,000
0
2,000
d
Firing rate (Hz)
1st
2nd
3rd
3rd to final 2nd to final
Final
24
24
24
24
24
24
12
12
12
12
12
12
0
–2,000
0
0
2,000 –2,000
0
0
2,000 –2,000
0
0
2,000 –2,000
Time (ms)
0
0
2,000 –2,000
0
0
2,000 –2,000
0
2,000
e
3rd
3rd to final 2nd to final
Final
30
30
30
30
30
30
15
15
15
15
15
15
1 6 12
Days of
training
0
2,000
80
40
40
40
0
0
0
Fi
rs
t
an Fin
d al
fin
al
0
k
80
rs
t
1 6 12
Days of
training
0
2,000 –2,000
Fi
0
0
Fi
rs
t
an Fin
d al
fin
al
1 6 12
Days of
training
25
j
80
rs
t
25
i
50
0
0
2,000 –2,000
Fi
0
h
0
0
2,000 –2,000
Time (ms)
Percentage
25
50
0
0
2,000 –2,000
Fi
rs
t
an Fin
d al
fin
al
g
Percentage
50
0
0
2,000 –2,000
rs
t
0
–2,000
f
2nd
Fi
Firing rate (Hz)
1st
Start/stop activity is action specific
We excluded the possibility that the emergence of start/stop activity
reflected interval timing24 (Supplementary Fig. 11) or stemmed
directly from the shortening of inter-press intervals and sequence
duration during training (Supplementary Fig. 12).
We also investigated whether this start/stop activity could be related
to different value expectations during the first and last presses of a
sequence compared to the middle presses25,26. We designed an experiment with two levers leading to different reward magnitudes under the
FR8 schedule (Methods). Mice perceived the different reward
amounts as indicated by the number of licks during the small and
large reward sessions (P , 0.01; Fig. 4a and Supplementary Fig. 13),
and learned the relationship between a particular action sequence and
the corresponding reward magnitude as indicated by their choice in
two-lever extinction tests (P , 0.01 for days 7 and 13; Fig. 4b).
We compared the firing rate modulation of each neuron in the
small versus large reward magnitude sessions (days 6 and 12; Fig. 4d,
g, j and Supplementary Fig. 14). Consistent with previous studies27,28,
dopaminergic neurons showed significantly higher firing rate modulation during lever pressing for a large versus a small reward
(8.0 6 1.7 Hz versus 5.2 6 1.0 Hz, P , 0.05; Fig. 4k). Striatal MSNs
(P . 0.05; Fig. 4e) and SN GABA neurons (P . 0.05; Fig. 4h) as a
population did not show different firing rate modulation during
pressing for large versus small rewards.
Importantly, the higher firing rate modulation of dopaminergic
neurons during large reward magnitude sessions was observed for
every press within a sequence (P , 0.05; Fig. 4l), but not particularly
during the first or last presses of a sequence (no differences in rate
modulation across different presses, P . 0.05). Similar results were
observed for striatal MSNs and SN GABA neurons (P . 0.05 for
both; Fig. 4f, i), even when considering only the neurons showing
preferential modulation by large or small reward magnitude
(Supplementary Fig. 15). These data indicate that sequence start/
stop-related activity cannot be easily attributed to differences in
reward expectation during the first or last presses of a sequence.
Because in this two-lever task mice are performing a similar
sequence of presses on each lever, we asked whether the start/stoprelated activity of each neuron was specific for a particular action (for
example, for initiating a sequence on the left but not on the right
lever), or generalized for similar actions. Most neurons with leverpress-related activity for both levers showed differential start/stop
activity between left and right lever sequences (67.8 6 8.5%,
74.2 6 3.4% and 70.8% for MSNs, SN GABA and SN DA neurons
respectively; similar across cell types, Kruskal–Wallis test, P . 0.05)
(Fig. 4c), indicating that sequence start/stop activity is action specific.
Disrupting start/stop activity impairs sequence learning
The learning of new sequences is compromised in disorders affecting
striatal circuits14–16. In the striatum, NMDA receptors are important for
dopamine-dependent plasticity at glutamatergic synapses29,30 and for
Figure 3 | Neural activity signalling the initiation and termination of action
sequences emerges in nigrostriatal circuits during learning. a–c, Example
of a striatal MSN (a), an SN GABA neuron (b) and an SN DA neuron
(c) showing significantly higher phasic firing selectively before the first press
of a sequence. d, Example of a striatal MSN showing a phasic increase in
activity preferentially before the last press of a sequence. e, Example of an SN
GABA neuron showing both sequence initiation and termination related
activity. Top, raster plot where each black dot indicates a spike and each
orange triangle represents a lever press; bottom, PETH for the same neuron
(a–e). From left to right, each column shows the PETH of the same neuron
related to the first, second, third, third to final, second to final and final press
of each sequence (a–e). f–h, Proportion of striatal MSNs (f), SN GABA
neurons (g) and SN DA neurons (h) showing sequence start/stop-related
activity (including only initiation, only termination or both) throughout
training. i–k, Proportion of striatal MSNs (i), SN GABA neurons (j) and SN
DA neurons (k) showing activity signalling sequence initiation, termination,
or both initiation and termination (day 12). Error bars denote s.e.m.
459
©2010 Macmillan Publishers Limited. All rights reserved
ARTICLES
NATURE | Vol 466 | 22 July 2010
6
0
Large reward
6
al
fin
al
Fi
na
l
to
fin
2n
d
3r
d
to
3r
d
0
Press in sequence
Figure 4 | Sequence start/stop-related activity does not reflect
differences in expected value and can be action specific. a, Total number of
licks for left and right single-lever FR8 forced-choice sessions, where the left
lever sequences led to a small reward and the right lever to a larger reward;
after six days the lever–reward-magnitude contingency was switched. b, Mice
prefer the lever leading to a larger reward during two-lever choice extinction
tests on day 7 and day 13. c, Percentage of striatal MSNs, SN GABA and SN
DA neurons showing action-specific start/stop-related activity. d, g, j, Firing
rate modulation of striatal MSNs (d), SN GABA neurons (g) and SN DA
neurons (j) in relation to lever pressing during small and large reward
sessions. e, h, k, The corresponding population average. f, i, l, The rate
modulation for each press within the action sequence for striatal MSNs
(f), SN GABA (i) and SN DA (l) neurons. Error bars denote s.e.m.
No. of lever press
1,000
Early Late
Stage of training
Percentage
2,980
No. of lever press
2,000
Time (s)
3,000
4,000
120
80
1,540
1,560
Time (s)
1,520
4
2
0
h 200
15
150
10
1
6
Days of training
k
0.8
0.6
0.4
0.2
0
20
1
6
Days of training
100
5
50
0
0
1.2
1.0
0.8
0.6
0.4
0.2
0
1
6
Days of training
l
1
6
Days of training
i
1.2
1.0
0.8
0.6
0.4
0.2
0
1
6
Days of training
Within-sequence
press rate (presses/min)
6
g
1,500
m
Within-sequence
press rate (CV)
CT
KO
1,000
Time (s)
ISl (s)
8
Duration of
sequence (s)
10
500
ISl (CV)
0
Duration
of sequence (CV)
Length of sequence (CV)
l
al
na
Fi
al
fin
to
2n
d
Small reward
2,970
Time (s)
80
Length of
sequence (presses)
l
al
na
Fi
al
fin
to
d
2n
3r
d
to
3r
d
fin
d
3r
d
12
Percentage
Lever presses
per min
No. of lever press
M
SN SN
G
AB
SN A
DA
to
3r
d
fin
t
Modulation rate (Hz)
12
Sm
0
0
10
20
30
40
Small reward modulation rate (Hz)
Press in sequence
Modulation rate (Hz)
10
La
Sm
20
Modulation rate (Hz)
30
j
l
k
al
lr
e
rg wa
r
e
re d
w
ar
d
Large reward
modulation rate (Hz)
40
80
2,960
160
12
6
0
40
Large reward
0
al
lr
ew
rg
ar
d
e
re
w
ar
d
0
15
30
45
60
Small reward modulation rate (Hz)
e
Press in sequence
0
0
j
1s
lr
ew
al
6
25
120
0
0
0
La
Sm
15
Modulation rate (Hz)
30
12
La
Large reward
modulation rate (Hz)
45
Early Late
Stage of training
50
160
80
Large reward
Small reward
c
CT
KO
40
0
1
6
Days of training
160
80
40
f
i
h
0
120
0
ar
d
re
w
ar
d
0
3
6
9
12
Small reward modulation rate (Hz)
60
Percentage
0
0
g
3
Small reward
t
3
3
6
rg
e
6
Modulation rate (Hz)
Large reward
modulation rate (Hz)
9
6
f
d
e
12
50
0
1s
d
0
7
13
Day of extinction test
10
120
2n
0
50
d
b
CT
KO
20
100
2n
1
6
12
Days of training
50
1s
t
500
c
100
100
2n
d
1,000
0
b
Modulation rate (Hz)
Left
Right
1,500
Percentage
2,000
Percentage
Total lick number
a
a
No. of lever press
the burst firing of MSNs31 (Supplementary Figs 16 and 17), as seen
during action sequence initiation and termination (Fig. 3a, d). We
investigated whether disrupting NMDA receptor function specifically
in the striatum would affect the development of sequence start/stop
activity in this structure. We used a Cre/loxP strategy to generate mutant
mice with a striatal-specific deletion of NMDAR1 (RGS9-Cre1/
NMDAR1-loxP2/2, referred to here as striatal NR1-knockout mice;
Methods)20, and recorded striatal activity in striatal NR1-knockout
and littermate controls during six days of FR8 training (n 5 16 for
knockout and 10 for control mice; Methods). Striatal NR1-knockout
mice learned to lever press to obtain sucrose (Fig. 5a), although there
were differences between genotypes (P , 0.01, mainly owing to longer
ISIs but not inter-press intervals in knockout mice; see later).
Similarly to what we observed in C57BL/6J mice (Fig. 2f), about 40%
of striatal MSNs showed lever-press-related activity in both striatal
NR1-knockout mice and littermate controls, and this proportion did
not change with training (P . 0.05; Fig. 5b and Supplementary Table
2). However, the percentage of neurons with start/stop activity was
significantly lower in striatal NR1-knockout mice than in controls
(P , 0.01; Fig. 5c), and did not increase with training in the mutants
1
6
Days of training
80
60
40
20
0
1.2
1.0
0.8
0.6
0.4
0.2
0
1
6
Days of training
1
6
Days of training
Figure 5 | Striatal-specific deletion of NMDA receptors disrupts the
development of start/stop activity and impairs sequence learning.
a, Average lever pressing rate per session during three days of CRF followed
by six days of FR8 training for striatal NR1-knockout (KO) mutants and
their littermate controls (CT). b, Proportion of MSNs in striatal NR1knockout mutants and littermate controls showing lever-press-related
activity during early and late stages of training. c, Proportion of MSNs in
striatal NR1-knockout mutants and littermate controls showing sequence
start/stop-related activity during the early and late stages of training.
d, e, Example of the behaviour microstructure of the same striatal NR1knockout mouse during the first day of FR8 training (d) and the sixth day of
FR8 training (e). All markers and insets are the same as those used in Fig. 1a,
b. f–i, Average sequence length (f), duration (g), ISI (h) and within-sequence
press rate (i) for mutants and littermate controls. j–m, CV of sequence length
(j), duration (k), ISI (l) and within-sequence press rate (m) during the first
and sixth day of FR8 training for striatal NR1-knockout mice and littermate
controls. Error bars denote s.e.m.
(P . 0.05) as it did in littermate controls (P , 0.05, similar to C57BL/
6J mice; Fig. 3f). Interestingly, this deficit was more apparent in
sensorimotor striatum (Supplementary Fig. 18).
Given the deficits in the development of start/stop activity in striatal
NR1-knockout mice, we examined sequence learning in mutants and
460
©2010 Macmillan Publishers Limited. All rights reserved
ARTICLES
NATURE | Vol 466 | 22 July 2010
littermate controls (n 5 14 for knockout and 19 for control mice;
Methods). We found that striatal NR1 knockouts showed little evidence of sequence learning compared with controls (Fig. 5d, e; see also
Fig. 1a, b), as indicated by similar sequence lengths between days 1 and
6 in knockout mice (P . 0.05; Fig. 5f). Also, unlike in controls, the
sequence length after training was significantly different from eight
presses in mutants (P , 0.01; Fig. 5f). Furthermore, the sequence duration also did not change from day 1 to day 6 in striatal NR1 knockouts
as it did in controls (P . 0.05 for knockout mice, P , 0.01 for controls;
Fig. 5g). This impairment in sequence learning did not stem from any
obvious motor impairments per se in striatal NR1-knockout mice,
because the within-sequence press rate was similar in knockout mice
and littermate controls (P . 0.05; Fig. 5i), and the ISI decreased with
training in knockout mice (P , 0.01; Fig. 5h). Importantly, the variability of sequence behaviour for each animal was generally higher in
knockout mice, and did not diminish as much with training as in
controls (for knockouts, sequence length, P , 0.01; sequence duration,
P . 0.05; ISI, P 5 0.05; within-sequence press rate, P . 0.05; for controls, P , 0.01 in all cases) (Fig. 5j–m).
Discussion
By investigating the behavioural microstructure and correlated
neural activity in a self-paced operant task, we found that neurons
in nigrostriatal circuits can signal the initiation and termination of
self-paced action sequences. This sequence start/stop neural activity
emerged as animals learned a specific action sequence, and was specific for particular actions. Importantly, this activity did not reflect
interval timing, changes in the inter-press interval during learning, or
differences in the expected value during the first or last actions of a
sequence. Furthermore, a striatal-specific manipulation that affects
plasticity and phasic firing in MSNs20 (Supplementary Fig. 17)
impaired the development of start/stop activity in the striatum and
selectively disrupted the learning of action sequences without affecting movement speed (Fig. 5i) or the ability to discriminate action
value (Supplementary Fig. 17). These data underscore the importance of the basal ganglia in learning and consolidating action
sequences7,8,32, and expand on previous studies showing changes in
striatal activity related to the initiation of cue-guided movements33–36, and sequence learning impairments in patients with focal
basal ganglia lesions16, Parkinson’s15 and Huntington’s diseases14.
Phasic firing of dopaminergic neurons has been widely studied as a
reward prediction error signal37,38, and our data confirm that phasic
firing of dopaminergic neurons before lever pressing can encode the
expected value of the outcome27,28. However, the possible function of
phasic dopamine signals in self-paced behaviour, namely in the initiation and termination of specific action sequences, or in the transition between different actions, has been somewhat neglected39. Our
data show that, in addition to striatal MSNs and SN GABA neurons,
the phasic activity of SN DA neurons can signal the initiation and
termination of specific action sequences. Hence, although the role of
dopamine in motor performance and in Parkinson’s has been mostly
associated with tonic dopamine38, our findings may have implications for the deficits in initiation and termination of voluntary movement sequences observed in Parkinson’s disease10–12.
These findings are therefore relevant for understanding the organization of actions and a variety of sequence learning and execution
deficits resulting from basal ganglia dysfunction11–16.
METHODS SUMMARY
twelve days on an FR8 schedule. For the two-lever, two-reward magnitude task,
every day animals had two single-lever sessions; pressing one lever led to a small
reward (15 ml) whereas pressing the other led to a large reward (50 ml). After six
days of training, the lever–reward-magnitude contingency was shifted. Fiveminute choice tests at the beginning of days 7 and 13 assessed the animals’
preference in extinction. For in vivo electrophysiology, tungsten microwire electrodes were implanted in the dorsal striatum and substantia nigra of naive animals
for chronic neural recording6 throughout operant training. Neural activity was
recorded simultaneously with the behavioural timestamps using the MAP system
(Plexon), and subjected to offline analyses. Analyses of neural activity were performed in Matlab (MathWorks) with custom-written programs. See Supplementary Information for further details.
Received 16 March; accepted 10 June 2010.
1.
2.
3.
4.
5.
6.
7.
8.
9.
10.
11.
12.
13.
14.
15.
16.
17.
18.
19.
20.
21.
22.
23.
All experiments were approved by the National Institute on Alcohol Abuse and
Alcoholism (NIAAA) Animal Care and Use Committee, the Portuguese Direcção
Geral de Veterinária, and done in accordance with National Institutes of Health
guidelines. C57BL/6J male mice (3 to approximately 6 months) were used in the
behavioural/recording experiments. Striatal-specific NR1-knockout mice and
control littermates were generated by crossing RGS9-Cre mice with NMDAR1loxP mice20. For behavioural training, mice were placed in an operant chamber
(Med-Associates) and trained to lever press to obtain 10% sucrose solution40.
Training started with three days of continuous reinforcement (CRF), followed by
24.
25.
26.
27.
Lashley, K. S. in Cerebral Mechanisms in Behavior (ed. Jeffress, L. A.) (John Wiley,
1951).
Gallistel, C. R. The Organization of Action: A New Synthesis (Lawrence Erlbaum
Associates, 1980).
Grillner, S. & Wallén, P. Central pattern generators for locomotion, with special
reference to vertebrates. Annu. Rev. Neurosci. 8, 233–261 (1985).
Marder, E. & Bucher, D. Central pattern generators and the control of rhythmic
movements. Curr. Biol. 11, R986–R996 (2001).
Hikosaka, O. et al. Parallel neural networks for learning sequential procedures.
Trends Neurosci. 22, 464–471 (1999).
Yin, H. H. et al. Dynamic reorganization of striatal circuits during the acquisition
and consolidation of a skill. Nature Neurosci. 12, 333–341 (2009).
Graybiel, A. M. The basal ganglia and chunking of action repertoires. Neurobiol.
Learn. Mem. 70, 119–136 (1998).
Hikosaka, O., Miyashita, K., Miyachi, S., Sakai, K. & Lu, X. Differential roles of the
frontal cortex, basal ganglia, and cerebellum in visuomotor sequence learning.
Neurobiol. Learn. Mem. 70, 137–149 (1998).
Bailey, K. R. & Mair, R. G. The role of striatum in initiation and execution of learned
action sequences in rats. J. Neurosci. 26, 1016–1025 (2006).
Benecke, R., Rothwell, J. C., Dick, J. P., Day, B. L. & Marsden, C. D. Disturbance of
sequential movements in patients with Parkinson’s disease. Brain 110, 361–379
(1987).
Agostino, R., Berardelli, A., Formica, A., Accornero, N. & Manfredi, M. Sequential
arm movements in patients with Parkinson’s disease, Huntington’s disease and
dystonia. Brain 115, 1481–1495 (1992).
Castiello, U., Stelmach, G. E. & Lieberman, A. N. Temporal dissociation of the
prehension pattern in Parkinson’s disease. Neuropsychologia 31, 395–402 (1993).
Phillips, J. G., Chiu, E., Bradshaw, J. L. & Iansek, R. Impaired movement sequencing
in patients with Huntington’s disease: a kinematic analysis. Neuropsychologia 33,
365–369 (1995).
Willingham, D. B. & Koroshetz, W. J. Evidence for dissociable motor skills in
Huntington’s disease patients. Psychobiology 21, 173–182 (1993).
Stefanova, E. D., Kostic, V. S., Ziropadja, L., Markovic, M. & Ocic, G. G. Visuomotor
skill learning on serial reaction time task in patients with early Parkinson’s disease.
Mov. Disord. 15, 1095–1103 (2000).
Boyd, L. A. et al. Motor sequence chunking is impaired by basal ganglia stroke.
Neurobiol. Learn. Mem. 92, 35–44 (2009).
Fujii, N. & Graybiel, A. M. Representation of action sequence boundaries by
macaque prefrontal cortical neurons. Science 301, 1246–1249 (2003).
Aldridge, J. W. & Berridge, K. C. Coding of serial order by neostriatal neurons: a
‘‘natural action’’ approach to movement sequence. J. Neurosci. 18, 2777–2787
(1998).
Meyer-Luehmann, M., Thompson, J. F., Berridge, K. C. & Aldridge, J. W. Substantia
nigra pars reticulata neurons code initiation of a serial pattern: implications for
natural action sequences and sequential disorders. Eur. J. Neurosci. 16, 1599–1608
(2002).
Dang, M. T. et al. Disrupted motor learning and long-term synaptic plasticity in
mice lacking NMDAR1 in the striatum. Proc. Natl Acad. Sci. USA 103, 15254–15259
(2006).
Gong, S. et al. Targeting Cre recombinase to specific neuron populations with
bacterial artificial chromosome constructs. J. Neurosci. 27, 9817–9823 (2007).
Boyden, E. S., Zhang, F., Bamberg, E., Nagel, G. & Deisseroth, K. Millisecondtimescale, genetically targeted optical control of neural activity. Nature Neurosci.
8, 1263–1268 (2005).
Lima, S. Q., Hromadka, T., Znamenskiy, P. & Zador, A. M. PINP: a new method of
tagging neuronal populations for identification during in vivo electrophysiological
recording. PLoS ONE 4, e6099, doi:10.1371/journal.pone.0006099 (2009).
Meck, W. H., Penney, T. B. & Pouthas, V. Cortico-striatal representation of time in
animals and humans. Curr. Opin. Neurobiol. 18, 145–152 (2008).
Samejima, K., Ueda, Y., Doya, K. & Kimura, M. Representation of action-specific
reward values in the striatum. Science 310, 1337–1340 (2005).
Lau, B. & Glimcher, P. W. Value representations in the primate striatum during
matching behavior. Neuron 58, 451–463 (2008).
Morris, G., Nevet, A., Arkadir, D., Vaadia, E. & Bergman, H. Midbrain dopamine
neurons encode decisions for future action. Nature Neurosci. 9, 1057–1063 (2006).
461
©2010 Macmillan Publishers Limited. All rights reserved
ARTICLES
NATURE | Vol 466 | 22 July 2010
28. Roesch, M. R., Calu, D. J. & Schoenbaum, G. Dopamine neurons encode the better
option in rats deciding between differently delayed or sized rewards. Nature
Neurosci. 10, 1615–1624 (2007).
29. Calabresi, P., Pisani, A., Mercuri, N. B. & Bernardi, G. Long-term potentiation in the
striatum is unmasked by removing the voltage-dependent magnesium block of
NMDA receptor channels. Eur. J. Neurosci. 4, 929–935 (1992).
30. Shen, W., Flajolet, M., Greengard, P. & Surmeier, D. J. Dichotomous dopaminergic
control of striatal synaptic plasticity. Science 321, 848–851 (2008).
31. Pomata, P. E., Belluscio, M. A., Riquelme, L. A. & Murer, M. G. NMDA receptor gating
of information flow through the striatum in vivo. J. Neurosci. 28, 13384–13389 (2008).
32. Brainard, M. S. & Doupe, A. J. What songbirds teach us about learning. Nature 417,
351–358 (2002).
33. Kimura, M. Behaviorally contingent property of movement-related activity of the
primate putamen. J. Neurophysiol. 63, 1277–1296 (1990).
34. Kermadi, I. & Joseph, J. P. Activity in the caudate nucleus of monkey during spatial
sequencing. J. Neurophysiol. 74, 911–933 (1995).
35. Jog, M. S., Kubota, Y., Connolly, C. I., Hillegaart, V. & Graybiel, A. M. Building
neural representations of habits. Science 286, 1745–1749 (1999).
36. Miyachi, S., Hikosaka, O. & Lu, X. Differential activation of monkey striatal
neurons in the early and late stages of procedural learning. Exp. Brain Res. 146,
122–126 (2002).
37. Schultz, W., Dayan, P. & Montague, P. R. A neural substrate of prediction and
reward. Science 275, 1593–1599 (1997).
38. Schultz, W. Multiple dopamine functions at different time courses. Annu. Rev.
Neurosci. 30, 259–288 (2007).
39. Redgrave, P. & Gurney, K. The short-latency dopamine signal: a role in discovering
novel actions? Nature Rev. Neurosci. 7, 967–975 (2006).
40. Hilário, M. R. F., Clouse, E., Yin, H. H. & Costa, R. M. Endocannabinoid signaling is
critical for habit formation. Front. Integr. Neurosci. 1, 6 (2007).
Supplementary Information is linked to the online version of the paper at
www.nature.com/nature.
Acknowledgements We thank Y. Li for the RGS9-Cre mice, C. Gerfen for the
TH-Cre mice, K. Nakazawa for the NMDAR1-loxP mice, F. Tecuapetla and S. Lima
for help in the optogenetics experiment, G. Luo for genotyping, and D. Lovinger,
G. Cui, C. French, C. Gremel and E. Dias-Ferreira for comments on the manuscript.
This research was supported by the NIAAA Division of Intramural Clinical and
Biological Research, the Champalimaud Neuroscience Programme at Institute
Gulbenkian de Ciência and European Research Council Grant 243393 to R.M.C.
Author Contributions X.J. performed the experiments and analysed the data.
R.M.C. conducted the optogenetics experiment. X.J. and R.M.C. designed the
experiments and wrote the paper.
Author Information Reprints and permissions information is available at
www.nature.com/reprints. The authors declare no competing financial interests.
Readers are welcome to comment on the online version of this article at
www.nature.com/nature. Correspondence and requests for materials should be
addressed to R.M.C. ([email protected]).
462
©2010 Macmillan Publishers Limited. All rights reserved