Chapter 12: Cognitive Control

THE AT TENDING PHYSICIAN in the neurology clinic had dealt with a lot of bizarre complaints during his 20 years of practice, but he was about to hear something new. W.R., a well-dressed man accompanied by his brother, complained to the neurologist that “he had lost his ego” (Knight & Grabowecky, 1995).

W.R. had been a focused kid, deciding while still a teenager that he wanted to become a lawyer. He remained focused on this goal in college, building up a nice GPA and resume. His life was well balanced: He found time for tennis, parties, and numerous girlfriends. After graduation, things continued just as planned. W.R. was admitted to the law school of his choice and completed the program with a solid, if not stellar, academic record. After earning his degree, however, his life suddenly seemed to change course. He no longer found himself driven to secure a job with a top law firm. Indeed, 4 years had passed and he still had not taken the bar exam or even looked for a job in the legal profession. Instead, he was an instructor at a tennis club.

His family was extremely disturbed to see the changes in W.R.’s fortunes. After law school, they thought he was experiencing an early midlife crisis that was not atypical of the times. They hoped that he would find satisfaction in his passion for tennis or that he would eventually resume his pursuit of a career in law. Neither had occurred. Indeed, he even gave up playing tennis. His opponents became frustrated because, shortly after commencing a match, W.R. would project an aura of nonchalance, forgetting to keep track of the score or even whose turn it was for service. Unable to support himself financially, he hit up his brother with increasingly frequent requests for “temporary” loans. As time passed, his family found it more and more difficult to tolerate W.R.’s behavior.

It was clear to the neurologist that W.R. was a highly intelligent man. He could clearly recount the many details of his life history, and he was cognizant that something was amiss. He realized that he had become a burden to his family, expressing repeatedly that he wished he could pull things together. He simply could not take the necessary steps, however, to find a job or get a place to live. His brother noted another radical change in W.R. Although he had been sexually active throughout his college years and had even lived with a woman, he had not been on a date for a number of years and seemed to have lost all interest in romantic pursuits. W.R. sheepishly agreed. He had little regard for his own future, for his successes, even for his own happiness. Though aware that his life had drifted off course, he just was not able to make the plans to effect any changes.

If this had been the whole story, the neurologist might have thought that a psychiatrist was a better option to treat a “lost ego.” Four years previously, however, during his senior year in law school, W.R. had suffered a seizure after staying up all night drinking coffee and studying for an exam. An extensive neurological examination done at the time, which included positron emission tomography (PET) and computer tomography (CT) scans, failed to identify the cause of the seizure. The neurologist was suspicious, however, given the claims of a lost ego combined with the patient’s obvious distractibility.

A CT scan that day confirmed the physician’s worst fears. W.R. had an astrocytoma. Not only was the tumor extremely large, but it had followed an unusual course. Traversing along the fibers of the corpus callosum, it had extensively invaded the lateral prefrontal cortex in the left hemisphere and a considerable portion of the right frontal lobe. This tumor had very likely caused the initial seizure, even though it was not detected at the time. Over the previous 4 years, it had slowly spread.

The next day, the neurologist informed W.R. and his brother of the diagnosis. Unfortunately, containment of the tumor was not possible. They could try radiation, but the prognosis was poor: W.R. was unlikely to live more than a year. His brother was devastated, shedding tears upon hearing the news. He had to face the loss of W.R., and he also felt guilty over the frustration he had felt with W.R.’s cavalier lifestyle over the previous 4 years. W.R., on the other hand, remained relatively passive and detached. Though he understood that the tumor was the culprit behind the dramatic life changes he had experienced, he was not angry or upset. Instead, he appeared unconcerned. He understood the seriousness of his condition; but the news, as with so many of his recent life events, failed to evoke a clear response or a resolve to take some action. W.R.’s self-diagnosis seemed to be right on target: He had lost his ego and, with it, the ability to take command of his own life.

Leaving the question of “the self” to Chapter 14, we can discern from W.R.’s actions, or rather inaction, that he had lost the ability to engage in goal-oriented behavior. Although he could handle the daily chores required to groom and feed himself, these actions were performed out of habit, without the context of an overriding goal, such as being prepared and full of energy to tussle in the legal system. He had few plans beyond satisfying his immediate needs, and even these seemed minimal. He could step back and see that things were not going as well for him as others hoped. But on a day-to-day basis, the signals that he was not making progress just seemed to pass him by.

In this chapter, our focus turns to the cognitive processes that permit us to perform more complex aspects of behavior. Cognitive control, or what is sometimes referred to as executive function, allows us to use our perceptions, knowledge, and goals to bias the selection of action and thoughts from a multitude of possibilities. Cognitive control processes allow us to override automatic thoughts and behavior and step out of the realm of habitual responses. They give us cognitive flexibility, letting us think and act in novel and creative ways. By being able to suppress some thoughts and activate others, we can simulate plans and consider the consequences of those plans. We can plan for the future and troubleshoot problems. Cognitive control is essential for purposeful goal-oriented behavior and decision making.

As we will see, the successful completion of goal-oriented behavior faces many challenges, and cognitive control is necessary to meet them. All of us must develop a plan of action that draws on our personal experiences, yet is tailored to the current environment. Such actions must be flexible and adaptive to accommodate unforeseen changes and events. We must monitor our actions to stay on target and attain that goal. Sometimes we need to constrain our own desires and follow rules to conform to social conventions. We may need to inhibit a habitual response in order to attain a goal. Although you might want to stop at the doughnut store when heading to work in the morning, cognitive control mechanisms can override that sugary urge, allowing you to stop by the café for a healthier breakfast.

The study of cognitive control brings us to a part of the cerebral cortex that has received little attention in preceding chapters—the prefrontal cortex. In this chapter, we concentrate on two prefrontal control systems (see the Anatomical Orientation box). The first, which includes the lateral prefrontal cortex and frontal pole, supports goal-oriented behavior. This control system works in concert with more posterior regions of the cortex to constitute a working memory system that recruits and selects task-relevant information. This system is involved with planning, simulating consequences, and initiating, inhibiting, and shifting behavior. The second control system, which includes the medial frontal cortex, plays an essential role in guiding and monitoring behavior. It works in tandem with the prefrontal cortex, monitoring ongoing activity to modulate the degree of cognitive control needed to keep behavior in line with goals. Before we get into these functions, we review some anatomy and consider cognitive control deficits that are seen in patients with frontal lobe dysfunction. We then focus on goal-oriented behavior and decision making, two complicated processes that rely on cognitive control mechanisms to work properly.

As might be suspected of any complex process, cognitive control requires the integrated function of many different parts of the brain. This chapter highlights the frontal lobes, and in particular, prefrontal cortex. The discussion, however, also requires references to other cortical and subcortical areas that are massively interconnected with the frontal cortex, forming the networks that enable goal-oriented behavior. This network includes the parietal lobe and the basal ganglia, regions that were discussed in previous chapters when we considered the neural mechanisms for attention and action selection.

The frontal lobes comprise about a third of the cerebral cortex in humans. The posterior border with the parietal lobe is marked by the central sulcus. The frontal and temporal lobes are separated by the lateral fissure.

As we learned in Chapter 8, the most posterior part of the frontal lobe is the primary motor cortex, encompassing the gyrus in front of the central sulcus and extending into the central sulcus itself. Anterior and ventral to the motor cortex are the secondary motor areas, including the lateral premotor cortex and the supplementary motor area. The remainder of the frontal lobe is termed the prefrontal cortex (PFC). The prefrontal cortex includes half of the entire frontal lobe in humans. The ratio is considerably smaller for non-primate species (Figure 12.1). We will refer to four regions of prefrontal cortex in this chapter: the lateral prefrontal cortex (LPFC), the frontal polar region (FP), the orbitofrontal cortex (OFC, or sometimes referred to as ventromedial zone), and the medial frontal cortex (MFC).

The frontal cortex is present in all mammalian species. In human evolution, however, it has expanded tremendously, especially in the more anterior aspects of prefrontal cortex. Interestingly, when compared to other primate species, the expansion of prefrontal cortex in the human brain is more pronounced in the white matter (the axonal tracts) than in the gray matter (the cell bodies; Schoenemann et al., 2005). This finding suggests that the cognitive capabilities that are uniquely human may be more a result of how our brains are connected rather than due to an increase in the number of neurons.

Because the development of functional capabilities parallels phylogenetic trends, the frontal lobe’s expansion is related to the emergence of the complex cognitive capabilities that are especially pronounced in humans. What’s more, as investigators frequently note, “Ontogeny recapitulates phylogeny.” Compared to the rest of the brain, prefrontal cortex matures late in terms of the development of neural density patterns and white matter tracts. Correspondingly, cognitive control processes appear relatively late in development, as evident in the “me-oriented” behavior of the infant and the rebellious teenager.

The prefrontal cortex coordinates processing across wide regions of the central nervous system (CNS). It contains a massively connected network that links the brain’s motor, perceptual, and limbic regions (Goldman-Rakic, 1995; Passingham, 1993). Extensive projections connect the prefrontal cortex to almost all regions of the parietal and temporal cortex, and even prestriate regions of the occipital cortex. The largest input comes from the thalamus, which connects the prefrontal cortex with subcortical structures including the basal ganglia, cerebellum, and various brainstem nuclei. Indeed, almost all cortical and subcortical areas influence the prefrontal cortex either through direct projections or indirectly via a few synapses. The prefrontal cortex also sends reciprocal connections to most areas that project to it, and to premotor and motor areas. It also has many projections to the contralateral hemisphere—projections to homologous prefrontal areas via the corpus callosum as well as bilateral projections to premotor and subcortical regions.

When we arrive at the discussion on decision making, which plays a prominent role in this chapter, we consider a finer-grained analysis of the dopamine system. This system includes the ventral tegmental area, a brainstem nucleus, the basal ganglia, and the dorsal and ventral striata (singular: striatum).

Patients with frontal lobe lesions like W.R., the wayward lawyer, present a paradox. From a superficial look at their everyday behavior, it is frequently difficult to detect a neurological disorder. They seem fine: They do not display obvious disorders in any of their perceptual abilities, they can execute motor actions, and their speech is fluent and coherent. These patients are unimpaired on conventional neuropsychological tests of intelligence and knowledge. They generally score within the normal range on IQ tests. Their memory for previously learned facts is fine, and they do well on most tests of long-term memory. With more sensitive and specific tests, however, it becomes clear that frontal lesions can disrupt different aspects of normal cognition and memory, producing an array of problems. Such patients may persist in a response even after being told that it is incorrect; this behavior is known as perseveration. These patients may be apathetic, distractible, or impulsive. They may be unable to make decisions, unable to plan actions, unable to understand the consequences of their actions, impaired in their ability to organize and segregate the timing of events in memory, unable to remember the source of their memories, and unable to follow rules—including a disregard of social conventions (discussed in the next chapter). Because the deficits seem to vary with the location of the patient’s lesion, it suggests that the neural substrates within the prefrontal cortex subserve different processes. As we’ll see, those processes are involved with cognitive control.

Ironically, patients with frontal lobe lesions are aware of their deteriorating social situation, have the intellectual capabilities to generate ideas that may alleviate their problems, and may be able to tell you the pros and cons of each idea. Yet their efforts to prioritize and organize these ideas into a plan and put them into play are haphazard at best. Similarly, though they are not amnesic, they are able to tell you a list of rules from memory, but may not be able to follow them.

A demonstration of how these problems are manifest in everyday behavior was given by Tim Shallice (Shallice & Burgess, 1991). He asked three patients with frontal lesions from head trauma to go to the local shopping center to make a few purchases (e.g., a loaf of bread), keep an appointment, or collect information such as the exchange rate of the rupee. These chores presented a real problem for the patients. For instance, one patient failed to purchase soap because the store she visited did not carry her favorite brand; another wandered outside the designated shopping center in pursuit of an item that could easily be found within the designated region. All became embroiled in social complications. One succeeded in obtaining the newspaper but was pursued by the merchant for failing to pay! In a related experiment, patients were asked to work on three tasks for 15 minutes. Whereas control participants successfully juggled their schedule to ensure that they made enough progress on each task, the patients got bogged down on one or two tasks.

Lesion studies in animals have revealed a similar paradox. Unilateral lesions of prefrontal cortex also tend to produce relatively mild deficits. When the lesions are bilateral, however, dramatic changes can be observed. Consider the observations of Leonardo Bianchi (1922), an Italian psychiatrist of the early 20th century:

The behavior of these monkeys underscores an important aspect of goal-oriented behavior. Following the lesions, the behavior is stimulus driven. The animal sees the ledge and jumps up; another sees the door and grasps the handle, but that is the end of it. They no longer appear to have a purpose for their actions. The sight of the door is no longer a sufficient cue to remind the animal of the food and other animals that can be found beyond it. The question is, what is the deficit? Is it a problem with motivation, attention, memory, or something else? Insightfully, Bianchi thought it was a problem with lack of representation in the “focal point of consciousness,” what we now think of as working memory.

A classic demonstration of this tendency for stimulus-driven behavior among humans with frontal lobe injuries is evident from the clinical observations of Francois Lhermitte of the Hôpital de la Salpêtrière in Paris (Lhermitte, 1983; Lhermitte et al., 1986). Lhermitte invited a patient to meet him in his office. At the entrance to the room, he had placed a hammer, a nail, and a picture. Upon entering the room and seeing these objects, the patient spontaneously used the hammer and nail to hang the picture on the wall. In a more extreme example, Lhermitte put a hypodermic needle on his desk, dropped his trousers, and turned his back to his patient. Whereas most people in this situation would consider filing ethical charges, the frontal lobe patient was unfazed. He simply picked up the needle and gave his doctor a healthy jab in the buttocks! Lhermitte coined the term utilization behavior to characterize this extreme dependency on prototypical responses for guiding behavior. The patients with frontal lobe damage retained knowledge about prototypical uses of objects such as a hammer or needle, saw the stimulus, and responded. They were not able to inhibit their response or flexibly change it according to the context in which they found themselves. Their cognitive control mechanisms were out of whack.

Our actions are not aimless, nor are they entirely automatic—dictated by events and stimuli immediately at hand. We choose to act because we want to accomplish a goal or gratify a personal need.

Researchers distinguish between two fundamental types of actions. Goal-oriented actions are based on the assessment of an expected reward or value and the knowledge that there is a causal relationship between the action and the reward (action–outcome). Most of our actions are of this type. We turn on the radio when getting into the car so that we can catch the news on the drive home. We put money into the soda machine to purchase a favorite beverage. We resist going out to the movies the night before an exam to get in some extra studying, with the hope that this effort will lead to the desired grade.

In contrast to goal-oriented actions stand habitual actions. A habit is defined as an action that is no longer under the control of a reward, but is stimulus driven; as such, we can consider it automatic. The habitual commuter might find herself flipping on the car radio without even thinking about the expected outcome. The action is triggered simply by the context. It becomes obvious that this is a habit when our commuter reaches to switch on the radio, even though she knows it is broken. Habit-driven actions occur in the presence of certain stimuli that trigger the retrieval of well-learned associations. These associations can be useful, allowing us to rapidly select a response (Bunge, 2004), such as stopping quickly at a red light. They can also develop into persistent bad habits, however, such as eating junk food when bored or lighting up a cigarette when anxious. Habitual responses make addictions difficult to break.

The distinction between goal-oriented behavior and habits is graded. Though the current context is likely to dictate our choice of actions and may even be sufficient to trigger a habitual-like response, we are also capable of being flexible. The soda machine might beckon invitingly, but if we are on a health kick, we might walk on past or choose to purchase a bottle of water. These are situations in which cognitive control comes into play.

Cognitive control provides the interface through which goals influence behavior. Goal-oriented behaviors require processes that enable us to maintain our goal, focus on the information that is relevant to achieving that goal, ignore or inhibit irrelevant information, monitor our progress toward the goal, and shift flexibly from one subgoal to another in a coordinated way.

As we learned in Chapter 9, working memory, a type of short-term memory, is the transient representation of task-relevant information—what Patricia Goldman-Rakic has called the “blackboard of the mind.” These representations may be from the distant past, or they may be closely related to something that is currently in the environment, or has been experienced recently. Working memory refers to the temporary maintenance of this information, providing an interface between perception, long-term memory, and action and thus, enabling goal-oriented behavior and decision making.

Working memory is critical for animals whose behavior is not exclusively stimulus driven. What is immediately in front of us surely influences our behavior, but we are not automatons. We can (usually) hold off eating until all the guests sitting around the table have been served. This capacity demonstrates that we can represent information that is not immediately evident, in this case social rules, in addition to reacting to stimuli that currently dominate our perceptual pathways (the fragrant food and conversation). We can mind our dinner manners (stored knowledge) by choosing to respond to some stimuli (the conversation) while ignoring other stimuli (the food). This process requires integrating current perceptual information with stored knowledge from long-term memory.

The lateral prefrontal cortex appears to be an important interface between current perceptual information and stored knowledge, and thus, constitutes a major component of the working memory system. Prefrontal cortex is necessary for cognitive control. Its importance in working memory was first demonstrated in animal studies using a variety of delayed-response tasks. In the simplest version, sketched in Figure 12.2, a monkey is situated within reach of two food wells. At the start of a trial, the monkey observes the experimenter placing a food morsel in one of the two wells (perception). Then the two wells are covered, and a curtain is lowered to prevent the monkey from reaching toward either well. After a delay period, the curtain is raised and the monkey is allowed to choose one of the two wells and recover the food. Although this appears to be a simple task, it demands one critical cognitive capability: The animal must continue to represent the location of the unseen food during the delay period (working memory). Monkeys with lesions of the lateral prefrontal cortex do poorly on the task.

The problem for these animals does not reflect a general deficit in forming associations. In an experiment to test associative memory, the food wells are covered with distinctive visual cues: The well with the food has a plus sign, and the empty well has a negative sign. In this condition, the researcher may shift the food morsel’s location during the delay period, but the associated visual cue—the food cover—will be relocated with the food. Prefrontal lesions do not disrupt performance in this task.

These two tasks clarify the concept of working memory (Goldman-Rakic, 1992). In the delayed-response task (see Figure 12.2a), the animal must remember the currently baited location during the delay period. In contrast, in the associative learning condition (see Figure 12.2b), it is only necessary for the visual cue to reactivate a long-term association of which cue is associated with the reward. The reappearance of the two visual cues can trigger recall and guide the animal’s performance.

Studies of patients with prefrontal lesions have also emphasized the role of this region in working memory. One example comes from studies of recency memory, the ability to organize and segregate the timing or order of events in memory (Milner, 1995). In a recency discrimination task, participants are presented with a series of study cards and every so often are asked which of two pictures was seen most recently. For example, one of the pictures might have been on a study card presented 4 trials previously, and the other, on a study card shown 32 trials back. For a control task, the procedure is modified: The test card contains two pictures, but only one of the two pictures was presented earlier. Following the same instructions, the participant should choose that picture because, by definition, it is the one seen most recently. Note, though, that the task is really one of recognition memory. There is no need to evaluate the temporal position of the two choices.

Patients with frontal lobe lesions perform as well as control participants on the recognition memory task, but they have a selective deficit in recency judgments. The memory task can be performed by evaluating if one of the stimuli was recently presented—or perhaps more relevant, if one of the stimuli is novel. The recency task, though, requires working memory in the sense that the patient must also keep track of the relationship between recently presented stimuli. This is not to suggest that the person could construct a full timeline of all of the stimuli—this would certainly exceed the capacity of working memory. But to compare the relative timing of two items, the frontal lobes are required to maintain the representations of those items at the time of the probe. When frontal lobes are damaged, this temporal structure is lost.

A breakdown in the temporal structure of working memory may account for more bizarre aspects of frontal lobe syndrome. For example, Wilder Penfield described a patient who was troubled by her inability to prepare her family’s evening meal. She could remember the ingredients for dishes and perform all of the actions to make the dish, but unless someone was there to tell her the proper sequence step by step, she could not organize her actions into a proper temporal sequence and could not prepare a meal (Jasper, 1995).

Another, albeit indirect, demonstration of the importance of prefrontal cortex in working memory comes from developmental studies. Adele Diamond of the University of Pennsylvania (1990) pointed out that a common marker of conceptual intelligence, Piaget’s Object Permanence Test, is logically similar to the delayed-response task. In this task, a child observes the experimenter hiding a reward in one of two locations. After a delay of a few seconds, the child is encouraged to find the reward. Children younger than 1 year are unable to accomplish this task. At this age, the frontal lobes are still maturing. Diamond maintained that the ability to succeed in tasks such as the Object Permanence Test parallels the development of the frontal lobes. Before this development takes place, the child acts as though the object is “out of sight, out of mind.” As the frontal lobes mature, the child can be guided by representations of objects and no longer requires their presence.

It seems likely that many species must have some ability to recognize object permanence. A species would not have survived for long if its members did not understand that a predator that had stepped behind a particular bush was still there. The difference between species may be in the capacity of the working memory, how long information can be maintained in working memory, and the ability to maintain attention (see the box How the Brain Works: Working Memory, Learning, and Intelligence).

A working memory system requires a mechanism to access stored information and a way to keep it active. The prefrontal cortex can perform both operations. In the delayed-response studies described earlier, single-cell recordings from the prefrontal cortex of monkeys (see Figure 12.3) showed that these neurons become active during the delayed-response task and show sustained activity throughout the delay period (Fuster, 1989). For some cells, activation doesn’t commence until after the delay begins and can be maintained up to 1 minute. These cells provide a neural correlate for keeping a representation active after the triggering stimulus is no longer visible. The cells provide a continuous record of the response required for the animal to obtain the reward.

Humans are obsessed with identifying why people differ in what we call “intelligence.” We have looked at anatomical measures such as brain size, prefrontal cortex size, amount of grey matter, and amount of white matter (connectivity). These measures have all been shown to account for some of the variation observed on tests of intelligence. Another approach is to consider differences among types of intelligence. For example, we can compare crystallized intelligence and ﬂuid intelligence. Crystallized intelligence refers to our knowledge, things like vocabulary and experience. Fluid intelligence refers to the ability to engage in creative abstract thinking, to recognize patterns, and to solve problems.

Fluid intelligence is closely linked to working memory. A child’s working memory at 5 years old turns out to be a better predictor of academic success than is IQ (Alloway & Alloway, 2010). Observations like these have inspired research on the neural mechanisms that are behind the differences in fluid intelligence. One study (Burgess et al., 2011) investigated whether the relationship between fluid intelligence and working memory is mediated by interference control, the ability to suppress irrelevant information. The researchers used fMRI while participants performed a classic working memory n-back task. Participants were presented with either word or face stimuli, and they were to respond when a stimulus matched one presented three items back. The researchers were curious about the neural response to “lures,” a stimulus that was recently presented but not 3-back (e.g., 2-back or 4-back). They assumed that the participants would show a tendency to respond to the lures and would need to exhibit interference control to suppress these responses. Indeed, an impressive positive correlation was found between the magnitude of the BOLD response to the lures in PFC (and parietal cortex) and fluid intelligence (Figure 1). They concluded that a key component of fluid intelligence is the ability to maintain focus on task-relevant information in working memory.

As we shall see in this chapter, the neurotransmitter dopamine plays an important role in learning. Dopamine receptors are abundant in PFC and thought to serve a modulatory function, sharpening the response of PFC neurons. It might be hypothesized that having more dopamine would predict better learning performance. Unlike fun, however, you can have too much dopamine. Various studies have shown that the efficacy of dopamine follows an inverted U-shaped function when performance is plotted as a function of dopamine levels. As dopamine levels increase, learning performance improves—but only to a point. At some level, increasing dopamine levels results in a reduction in performance.

The inverted U-shaped function can help explain some of the paradoxical effects of L-dopa therapy in Parkinson’s disease. In these patients, the reduction in dopamine levels is most pronounced in dorsal (motor) striatum, at least in the early stages of the disease; dopamine levels in ventral striatum and the cerebral cortex are less affected. L-dopa treatment boosts dopamine levels back to normal in the dorsal striatum and thus improves motor function. The same treatment, however, produces an overdose effect in the ventral striatum and frontal lobe. This can result in impaired performance on tasks that depend on ventral striato-frontal circuitry such as reversal learning, where you have to change your behavior to gain a reward (Graef & Heekeren, 2010).

Genetic data shows that different alleles can affect dopamine levels, which in turn have an effect on PFC function. The catecholamine-O-methyltransferase (COMT) gene is associated with the production of an enzyme that breaks down dopamine. There are different alleles of COMT, resulting in different levels of the enzyme. People with the allele that lowers the rate of dopamine breakdown have higher dopamine levels, especially in the PFC. Interestingly, this allele has been implicated in an increased risk for schizophrenia and other neuropsychiatric phenotypes.

FIGURE 1 Correlation of control network activity and measures of fluid intelligence.
Participants performed a working memory task. Trials were divided into those with lures where a mismatch was a stimulus that had been previously seen (and thus had potential for a false alarm) and trials without lures where the stimulus had not been seen. (a) Regions in prefrontal and parietal cortex that had increased BOLD response on lure trials compared to no-lure trials. (b) Correlation between individual scores on the Lure activity factor (Lure–No Lure) and measures of fluid intelligence (left) or a measure of working memory span from a different task (right).

Lateral prefrontal cortex (LPFC) cells simply could be providing a generic signal that supports representations in other cortical areas. Alternatively, they could be coding specific stimulus features. To differentiate between these possibilities, Earl Miller and his colleagues (Rao et al., 1997) trained monkeys on a working memory task that required successive coding of two stimulus attributes: identity and location. Figure 12.4a depicts the sequence of events in each trial. A sample stimulus is presented, and the animal must remember the identity of this object for a 1-s delay period in which the screen is blank. Then two objects are shown, one of which matches the sample. The position of the matching stimulus indicates the target location for a forthcoming response. The response, however, must be withheld until the end of a second delay. Within the lateral prefrontal cortex, cells characterized as “what,” “where,” and “what–where” were observed (Figure 12.4). For example, “what” cells responded to specific objects, and this response was sustained over the delay period. “Where” cells showed selectivity to certain locations. In addition, about half of the cells were “what–where” cells, responding to specific combinations of “what” and “where” information. A cell of this type exhibited an increase in firing rate during the first delay period when the target was the preferred stimulus. Moreover, the same cell continued to fire during the second delay period if the response was directed to a specific location.

These results indicate that, in terms of stimulus attributes, cells in the prefrontal cortex exhibit taskspecific selectivity. What’s more, the activity of these PFC cells is dependent on the monkey using that information to obtain a response. That is, the activity of the PFC cells is task-dependent. If the animal only has to passively view the stimuli, then the response of these cells is minimal right after the stimulus is presented and entirely absent during the delay period. Moreover, the response of these cells is malleable. If the task conditions change, the same cells become responsive to a new set of stimuli (Freedman et al., 2001).

These cellular responses by themselves do not tell us what is represented by this protracted activity. It could be that long-term representations are stored in the prefrontal cortex, and the activity reflects the need to keep these representations active during the delay. Patients with frontal lobe lesions do not have deficits in long-term memory, however, so this hypothesis is unlikely. An alternative hypothesis is that prefrontal activation reflects a representation of the task goal, and as such, serves as an interface with task-relevant long-term representations in other neural regions (Figure 12.5). This latter hypothesis jibes nicely with the fact that the prefrontal cortex is extensively connected with postsensory regions of the temporal and parietal cortex. When a stimulus is perceived, a representation can be sustained through the interactions between prefrontal cortex and posterior brain regions, one that can facilitate goal-oriented behavior.

This alternative hypothesis has been examined in many functional imaging studies. In one representative study, researchers used a variant of a delayed-response task (Figure 12.6a). On each trial, four stimuli were presented successively for 1 s each during an encoding interval. The stimuli were either intact faces or scrambled faces. The participants were instructed to remember only the faces. Thus, by varying the number of intact faces presented during the encoding interval, the processing demands on working memory were manipulated. After an 8-s delay, a face stimulus—the probe—was presented, and the participant had to decide if the probe matched one of the faces presented during the encoding period. The BOLD response in the lateral prefrontal cortex bilaterally began to rise with the onset of the encoding period, and this response was maintained across the delay period even though the screen was blank (Figure 12.6b). This prefrontal response was sensitive to the demands on working memory. The sustained response during the delay period was greater when the participant had to remember three or four intact faces as compared to just one or two intact faces.

FIGURE 12.6 Functional MRI study of working memory.
(a) In a delayed-response task, a set of intact faces or scrambled faces is presented during an encoding period. After a delay period, a probe stimulus is presented and the participant indicates if that face was part of the memory set. (b) The BOLD response in lateral prefrontal cortex (PFC) rises during the encoding phase and remains high during the delay period. The magnitude of this effect is related to the number of faces that must be maintained in working memory. (c) The BOLD response in the lateral prefrontal cortex and the fusiform face area (FFA) rises during the encoding and retrieval periods. The black dotted and red dotted lines indicate the peak of activation in the FFA and PFC. During encoding, the peak is earlier in the FFA; during retrieval, the peak is earlier in the PFC.

By using faces, the experimenters could also compare activation in the prefrontal cortex with that observed in the fusiform face area, the inferior temporal (also called the inferotemporal) lobe region that was discussed in Chapter 6. The BOLD responses for these two regions are shown in Figure 12.6c, where the data are combined over the different memory loads. When the stimuli were presented, either during the encoding phase or for the memory probe, the BOLD response was much stronger in the FFA than in the prefrontal cortex. During the delay period, as noted already, the prefrontal response remains high. Note, however, that although a substantial drop in the FFA BOLD response occurs during the delay period, the response does not drop to baseline, thus suggesting that this area continues to be active during the delay period. In fact, the BOLD response in other perceptual areas of the inferior temporal cortex actually goes below baseline—the so-called rebound effect. Thus, although the sustained response is small in the FFA, it is considerably higher than what would be observed with nonfacial stimuli.

The timing of the peak activation in the prefrontal cortex and the FFA is also intriguing. During encoding, the peak response is slightly earlier in the FFA as compared to the prefrontal cortex. In contrast, during memory retrieval the peak response is slightly earlier in the prefrontal cortex. Although this study does not allow us to make causal inferences, the results are consistent with the general tenets of the model sketched in Figure 12.5. Lateral prefrontal cortex is critical for working memory by sustaining a representation of the task goal (to remember faces) and working in concert with inferotemporal cortex to sustain information across the delay period that is relevant for achieving that goal.

Working memory is necessary for keeping task-relevant information active as well as manipulating that information to accomplish behavioral goals. Think about what happens when you reach for your wallet to pay a bill after dinner at a restaurant. Besides the listed price, you have to remember to add a tip, drawing on your long-term knowledge of the appropriate behavior in restaurants. With this goal in mind, you then do some fancy mental arithmetic (if your cell phone calculator isn’t handy). Michael Petrides (2000) suggests a model of working memory, in which information held in the posterior cortex is activated, retrieved, and maintained by the ventrolateral PFC (e.g., the standard percentage for a tip) and then manipulated with the relevant information (e.g., the price of the dinner) in more dorsal regions of lateral PFC, enabling successful attainment of the goal.

Let’s consider the experimental setup shown in Figure 12.7a. Memory was probed in two different ways. When the judgment required that the items be remembered in the forward direction, study participants had to internally maintain a representation of the stimuli using a natural scheme in which the items could be rehearsed in the order presented. The backward conditions were more challenging, because the representations of the items had to be remembered and manipulated. As Figure 12.7b shows, the BOLD response was similar in ventral prefrontal region (ventrolateral PFC) for all conditions. In contrast, the response in dorsolateral PFC was higher for the two backward conditions.

A similar pattern is observed in many imaging studies, indicating that dorsolateral prefrontal cortex is critical when the contents of working memory must be manipulated. One favorite testing variant for studying manipulations in working memory is the n-back task (Figure 12.8). Here the display consists of a continuous stream of stimuli. Participants are instructed to push a button when they detect a repeated stimulus. In the simplest version (n = 1), responses are made when the same stimulus is presented on two successive trials. In more complicated versions, n can equal 2 or more. With n-back tasks, it is not sufficient simply to maintain a representation of recently presented items; the working memory buffer must be updated continually to keep track of what the current stimulus must be compared to. Tasks such as n-back tasks require both the maintenance and the manipulation of information in working memory. Activation in the lateral prefrontal cortex increases as n-back task difficulty is increased, a response consistent with the idea that this region is critical for the manipulation operation.

The n-back tasks capture an essential aspect of prefrontal function, emphasizing the active part of working memory. The recency task that patients with frontal lobe lesions fail also reflects the updating aspect of working memory. That task requires the participants to remember whether something is familiar as well as the context in which the stimulus was previously encountered.

The maintain-manipulate distinction offers a processing-based account of difference in PFC function along a ventral-dorsal gradient. This idea was primarily based on activation differences in more posterior regions of PFC. But what about the most anterior region, the frontal pole? The study comparing verbal and spatial working memory provides one hint. The frontal pole was the one region that was recruited upon presentation of the instruction cue in all four conditions (shown in Figure 12.7) and that maintained a high level of activity throughout the trial. It has been proposed that the frontal pole is essential for integrating the specific contents of mental activity into a general framework. Consider that the participants in the scanner had to remember to study the test items and other details, such as staying awake, not moving, and responding quickly. They also had to remember the big picture, the context of the test: They had volunteered to participate in an fMRI study with hopes of doing well so that they could volunteer to work in the lab and run their own cool experiments in the future.

Work focused on the anterior–posterior gradient across the PFC suggests that activation patterns follow a crude hierarchy: For the simplest of working memory tasks, the activation may be limited to more posterior prefrontal regions or even secondary motor areas. For example, if the task requires the participant to press one key upon seeing a flower and another upon seeing an automobile, then these relatively simple stimulus–response rules can be sustained by the ventral prefrontal cortex and lateral premotor cortex. If the stimulus–response rule, however, is defined not by the objects themselves but by a color surrounding the object, then the frontal pole is also recruited (Bunge, 2004). When such contingencies are made even more challenging by changing the rules from one block of trials to the next, activation extends even farther in the anterior direction (Figure 12.9). These complex experiments demonstrate how goal-oriented behavior can require the integration of multiple pieces of information.

As a heuristic, we can think of PFC function as organized along three separate axes (see O’Reilly, 2010):

For example, suppose it is the hottest day of summer and you are at the lake. You think, “It would be great to have a frosty, cold drink.” This idea starts off as an abstract desire, but then is transformed into a concrete idea as you remember root beer floats from summer days past. This transformation entails a spread in activation from the most anterior regions of PFC to medial regions as orbitofrontal cortex helps with the recall of the high value you associate with your previous encounters with root beer floats. More posterior regions become active as you begin to develop an action plan. You become committed to the root beer float, and that goal becomes the center of working memory and thus engages lateral prefrontal cortex. You think about how good these drinks are at A&W restaurants, drawing on links from more ventral regions of PFC to long-term memories associated with their floats. You also draw on dorsal regions that will be essential for forming a plan of action to drive to the A&W. It’s a complicated plan, one that no other species would come close to accomplishing. Luckily for you, however, your PFC network is buzzing along now, highly motivated, with the ability to establish the sequence of actions required to accomplish your goal. Reward is just down the road.

Go back to the hot summer day when you thought, “Hmm ... that frosty, cold drink is worth looking for. I’m going to get one.” That type of goal-oriented behavior begins with a decision to pursue the goal. We might think of the brain as a decision-making device whose perceptual, memory, and motor capabilities evolved to support decisions that determine actions. Our brains start making decisions as soon as our eyes flutter open in the morning: Do I get up now, or stay cozy and warm for another hour? Should I surf my email or look over my homework before class? Do I skip class to take off for a weekend ski trip? Though humans tend to focus on complex decisions such as who gets their vote in the next election, all animals need to make decisions. Even an earthworm decides when to leave a patch of lawn and move on to greener pastures.

Rational observers, such as economists and mathematicians, tend to be puzzled when they consider human behavior. To them, our behavior frequently appears inconsistent or irrational, not based on what seems to be a sensible evaluation of the circumstances and options. For instance, why would someone who is concerned about eating healthy food eat a jelly doughnut? Why would someone who is paying so much money for tuition skip classes?—a question that your non-economist parents might even ask. And why are people willing to spend large sums of money to insure themselves against low-risk events (e.g., buying fire insurance even though the odds are overwhelmingly small that they will ever use it), yet are willing to engage in high-risk behaviors (e.g., driving after consuming alcohol)?

The field of neuroeconomics has emerged as an interdisciplinary enterprise with the goal of explaining the neural mechanisms underlying decision making. Economists want to understand how and why we make the choices we do. Many of their ideas can be tested both with behavioral studies and, as in all of cognitive neuroscience, with data from cellular activity, neuroimaging, or lesion studies. This work also helps us understand the functional organization of the brain.

Theories about our decision-making processes are either normative or descriptive. Normative decision theories define how people ought to make decisions that yield the optimal choice. Very often, however, such theories fail to predict what people actually choose. Descriptive decision theories attempt to describe what people actually do, not what they should do. Our inconsistent, sometimes suboptimal choices present less of a mystery to evolutionary psychologists. Our modular brain has been sculpted by evolution to optimize reproduction and survival in a world that differed quite a bit from the one we currently occupy. In that world, you might never pass up the easy pickings of a jelly doughnut (that is, something sweet and full of fat), and you would not engage in exercise solely for the sake of burning off valuable calories; conserving energy would likely be a much more powerful factor. Our current brains reflect this past, drawing on the mechanisms that were essential for survival in a world before readily available junk food. Many of these mechanisms, as with all brain functions, putter along below our consciousness. We are unaware that many of our decisions are made following simple, efficient rules (heuristics) that were sculpted and hard-coded by evolution. The results of these decisions may not seem rational, at least within the context of our current, highly mechanized world. But they may seem more rational if looked at from an evolutionary perspective.

Consistent with this point of view, the evidence indicates that we reach decisions in many different ways. As we touched on earlier, decisions can be goal oriented or habitual. The distinction is that goal-oriented decisions are based on the assessment of expected reward, whereas habits, by definition, are actions taken that are no longer under the control of reward: We simply execute them because the context triggers the action. A somewhat similar way of classifying decisions is dividing them into action– outcome decisions or stimulus–response decisions. With an action–outcome decision, the decision involves some form of evaluation (not necessarily conscious) of the expected outcomes. After we repeat that action, and if the outcome is consistent, the process becomes habitual; that is, it becomes a stimulus–response decision. Another distinction can be made between decisions that are model-free or model-based. Model-based means that the agent has an internal representation of some aspect of the world and uses this model to evaluate different actions. For example, a cognitive map would be a model of the spatial layout of the world, allowing you to choose an alternative path if you find the road blocked as you set off for the A&W restaurant. Model-free means that you just have an input–output mapping, similar to stimulus–response decisions. Here you know that to get to the A&W, you simply look for the tall tower at the center of town, knowing that the A&W is right next door. Decisions that involve other people are known as social decisions. Dealing with other individuals tends to make things much more complicated, a topic we will return to in Chapters 13 and 14.

A cornerstone idea in economic models of decision making is that before we make a decision, we first compute the value of each of the options and then make some sort of comparison of the different values (Padoa-Schioppa, 2011). Decision making in this framework is about making choices that will maximize value. For example, we want to obtain the highest possible reward or payoff (Figure 12.10). It is not enough, however, to just think about the possible reward level. We also have to consider the likelihood of receiving the reward, as well as the costs required to obtain that reward. Although many lottery players dream of winning the $1,000,000 prize, some may forgo a chance at the big money and buy a ticket with a maximum payoff of $100, knowing that their odds are much higher.

To understand the neural processes involved in decision making, we need to understand how the brain computes value and processes rewards. Some rewards, such as food, water, or sex, are primary reinforcers: They have a direct benefit for survival fitness. Their value, or our response to these reinforcers, is to some extent hardwired in our genetic code. But reward value is also flexible and shaped by experience. If you are truly starving, an item of disgust—say, a dead mouse—suddenly takes on reinforcing properties. Secondary reinforcers, such as money and status, are rewards that have no intrinsic value themselves but become rewarding through their association with other forms of reinforcement.

Reward value is not a simple calculation. Value has various components, both external and internal, that are integrated to form an overall subjective worth. Consider this scenario: You are out fishing along the shoreline and thinking about whether to walk around the lake to an out-of-the-way fishing hole. Do you stay put or pack up your gear? Establishing the value of these options requires considering several factors, all of which contribute to the representation of value:

Payoff: What kind and how much reward do the options offer? At the current spot, you might land a small trout or perhaps a bream. At the other spot, you’ve caught a few large-mouthed bass.

Probability: How likely are you to attain the reward? You might remember that the current spot almost always yields a few catches, whereas you’ve most often come back empty-handed from the secret hole.

Effort or cost: If you stay put, you can start casting right away. Getting to the fishing hole on the other side of the lake will take an hour of scrambling up and down the hillside. One form of cost that has been widely studied is temporal discounting. How long are you willing to wait for a reward? You may not catch large fish at the current spot, but you could feel that satisfying tug 30 minutes sooner if you stay where you are.

Context: This factor involves external things, like the time of day, as well as internal things, such as whether you are hungry or tired, or looking forward to an afternoon outing with some friends. Context also includes novelty—you might be the type who values an adventure and the possibility of finding an even better fishing hole on your way to the other side of the lake, or you might be feeling cautious, eager to go with a proven winner.

Preference: You may just like one fishing spot better than another for its aesthetics or a fond memory.

As you can see, there are many factors that contribute to subjective value, and they can change immensely from person to person and hour to hour. Given such variation, it is not so surprising that people are highly inconsistent in their decision-making behavior. What seems irrational thinking by another individual might not be, if we could peek into that person’s up-to-date value representation of the current choices.

How and where is value represented in the brain? Is it all in one place or represented separately? Multiple single-cell studies and fMRI studies have been conducted to examine this question. Jon Wallis and his colleagues (Kennerley et al., 2009) looked at value representation in the frontal lobes of monkey brains. They targeted this region because damage to the frontal lobes is associated with impairments in decision making and goal-oriented behavior. While the monkey performed decision-making tasks, the investigators used multiple electrodes to record from cells in three regions: the anterior cingulate cortex (ACC), the lateral prefrontal cortex (LPFC), and the orbitofrontal cortex (OFC). Besides comparing cellular activity in different locations, the experimenters manipulated cost, probability, and payoff. The key question was whether the different areas would show selectivity to particular dimensions. For instance, would OFC be selective to payoff, LPFC to probability, and ACC to effort? Or, would they find an area that coded overall “value” independent of the variable?

The monkey’s decision involved choosing between two pictures (Figure 12.11). Each picture was associated with a specific value on one of the three dimensions: cost of payoff, amount of payoff, and probability of payoff. For instance, the guitar was always associated with a payoff of three drops of juice, and the flower was always associated with a two-drop juice payoff. On any given trial, choice value was manipulated along a single dimension while holding the other two dimensions constant. So, for example, when cost (the number of lever presses necessary to gain a reward) varied between the pictures, the probability and the payoff (a specific amount of juice) were constant. Each area included cells that responded to each dimension as well as cells that responded to multiple dimensions. Interestingly, there was no clear segregation between the representations of these three dimensions of value. There were some notable differences, however. First, many cells—especially in ACC, but also in the OFC—responded to all three dimensions. A pattern like this suggests these cells represent an overall measure of value. In contrast, LPFC cells usually encoded just one decision variable. Second, if cells represented one variable only, it was usually payoff or probability. If cells responded to effort, then they usually responded to something else also. The value signal in the cells’ activity generally preceded that associated with motor preparation, suggesting that choice value is being computed before the response is selected—an observation that makes intuitive sense.

Similar studies with human participants have been conducted with fMRI. Here the emphasis has been on how different dimensions preferentially activate different neural regions. For example, in one study, OFC activation was closely tied to variation in payoff, whereas activation in the striatum of the basal ganglia was related to effort (Croxson et al., 2009). In another study, lateral PFC activation was associated with the probability of reward, whereas the delay between the time of the action and the payoff was correlated with activity in medial PFC and lateral parietal lobe (Peters & Buchel, 2009).

What happens in the all-too-common situation when a payoff has high short-term value but adverse long-term value? For example, what happens when dieters are given their choice of a tasty but unhealthy treat (like the jelly doughnut) and a healthy but perhaps less tasty one (the yogurt breakfast)? Interestingly, activity in ventromedial prefrontal cortex (VMPFC), including OFC, was correlated with taste preference, regardless of whether the item was healthy (Figure 12.12). In contrast, the dorsolateral prefrontal cortex (DLPFC) area was associated with the degree of control (Hare et al., 2009). Activity here was greater on trials in which a preferred, but unhealthy, item was refused compared to when that item was selected. Moreover, this difference was much greater in people who were judged to be better in exhibiting self-control. It may be that the VMPFC originally evolved to forecast the short-term value of stimuli. Over evolutionary time, structures such as the DLPFC began to modulate more primitive, or primary, value signals, providing humans with the ability to incorporate long-term considerations into value representations. These findings also suggest that a fundamental difference between successful and failed self-control might be the extent to which the DLPFC can modulate the value signal encoded in the VMPFC (see Chapter 13 for a related take on VMPFC).

Overall, the neurophysiological and neuroimaging studies indicate that OFC plays a key role in the representation of value. More lateral regions of the PFC are important for some form of modulatory control on these representations or the actions associated with them. We have seen one difference between the neurophysiological and neuroimaging results: The former emphasize a distributed picture of value representation, and the latter emphasize specialization within components of a decision-making network. The discrepancy, though, is likely due to the differential sensitivity of the two methods. The fine-grained spatial resolution of neurophysiology allows us to ask if individual cells are sensitive to particular dimensions. In contrast, fMRI studies generally provide relative answers, asking if an area is more responsive to variation in one dimension compared to another dimension.

The laboratory is an artificial environment. Many of the experimental paradigms used to study decision making involve conditions in which the participant has ready access to the different choices and at least some information about the potential rewards and costs. Thus, participants are in a position where they can calculate and compare values. In the natural environment, especially the one our ancestors roamed about in, this situation is the exception and not the norm. Rather, we frequently face situations where we must choose between an option with a known value and one or more options of unknown value (Rushworth et al., 2012). The classic example here is foraging: Animals have to make decisions about where to seek food and water, precious commodities that tend to occur in restricted locations and for only a short time. Foraging requires decisions such as, “Do I keep eating/hunting/fishing here or move on to (what may or may not be) greener pastures, birdier bushes, or fishier water holes?” In other words, do I continue to exploit the resources at hand or set out to explore in hopes of finding a richer niche? Here the animal must calculate the value of the current option, richness of the overall environment, and the costs of exploration.

Worms, bees, wasps, spiders, fish, birds, seals, monkeys, and human subsistence foragers all obey a basic principle in their foraging behavior. This principle is referred to as the marginal value theorem (Charnov, 1974). The animal exploits a foraging patch until its intake rate falls below the average intake rate for the overall environment. At that point, the animal becomes exploratory. Because this behavior is so consistent across so many species, scientists have hypothesized that this tendency may be deeply encoded in our genes. Indeed, biologists have identified a specific set of genes that influence how worms decide when it is time to start looking for “greener lawns” (Bendesky et al., 2011).

Benjamin Hayden (2011) and his colleagues investigated the neural mechanisms that might be involved in foraging-like decisions. They hypothesized that such decisions require a decision variable, a representation that specifies the current value of leaving a patch, even if the alternative choice is relatively unknown. When this variable reaches a threshold, a signal is generated indicating that it is time to look for greener pastures. A number of factors influence how soon this threshold is reached: the current expected payoff, the expected benefits and costs for traveling to a new patch, and the uncertainty of obtaining reward at the next location. For instance, in our fishing example, if it takes two hours instead of one to go around the lake to a better fishing spot, you are less likely to move.

Hayden recorded from cells in the ACC of monkeys, choosing this region because it has been linked to the monitoring of actions and their outcomes (which we discuss later in the chapter). The animals were presented with a virtual foraging task in which they chose one of two targets. One stimulus was followed by a reward after a short delay, but the amount decreased with each successive trial (equivalent to remaining in a patch and reducing the food supply by eating it). The other stimulus allowed the animals to change the outcome contingencies. They received no reward on that trial, but after waiting for a variable period of time (the cost of exploration), the choices were presented again and the payoff for the rewarded stimulus was reset to its original value (a greener patch). Consistent with the marginal value theorem, they were less likely to choose the rewarded stimulus as the waiting time increased or the amount of reward decreased. What’s more, the cellular activity in ACC was highly predictive of the amount of time the animal would continue to “forage” by choosing the rewarding stimulus. Most interesting, the cells showed the property of a threshold: When the firing rate was greater than 20 spikes per second, the animal left the patch (Figure 12.13).

The hypothesis that the ACC plays a critical role in foraging-like decisions is further supported by fMRI studies with humans (Kolling et al., 2012). When the person is making choices about where to sample in a virtual world, the BOLD response in ACC correlates positively with search value (explore) and negatively with the encounter value (exploit) regardless of which choice participants made. In this condition, ventromedial regions of the PFC did not signal overall value. If, however, experimenters modified the task so that the participants were engaged in a comparison decision, activation in VMPFC reflected the chosen option value. Taken together, these studies suggest that ACC signals exert a type of control by promoting a particular behavior: exploring the environment for better alternatives compared to the current course of action (Rushworth et al., 2012).

We have seen that rewards, especially those associated with primary reinforcers like food and sex, are fundamental to the behavior of all animals. It follows that we might expect the processing of such signals to involve phylogenetically older neural structures. Indeed, converging lines of evidence indicate that many subcortical areas represent reward information, including the dorsal and ventral striatum, hypothalamus, amygdala, and lateral habenula (for a review, see Hikosaka et al., 2008). Much of the work on reward has focused on the neurotransmitter dopamine. We should keep in mind, however, that reinforcement likely involves the interplay of many transmitters. For instance, evidence suggests that serotonin is important for temporal discounting of reward value (Tanaka et al., 2007).

Dopaminergic cells are scattered throughout the midbrain, sending axonal projections to many cortical and subcortical areas. Two of the primary loci of dopaminergic neurons are two brainstem nuclei, the substantia nigra pars compacta (SNc) and ventral tegmental area (VTA). As discussed in Chapter 8, the dopaminergic neurons from the substantia nigra project to the dorsal striatum, the major input nucleus of the basal ganglia. Loss of these neurons is related to the movement initiation problems observed in patients with Parkinson’s disease. Dopaminergic neurons that originate in the VTA project through two pathways. The mesolimbic pathway travels to structures important to emotional processing, including the nucleus accumbens (ventral striatum) of the basal ganglia, the amygdala, the hippocampus, and the anterior cingulate cortex. The VTA also has dopaminergic projections that travel through the mesocortical pathway to the neocortex, particularly to the medial portions of the frontal lobe.

The link between dopamine and reward began with the classic work of James Olds and Peter Milner in the early 1950s (Olds, 1958; Olds & Milner, 1954). They implanted electrodes into the brains of rats and then gave the rats the opportunity to control the electrodes. When the rat pushed a lever, the electrode became activated. Some of the rats rarely pressed the lever. Others pressed the lever like crazy. The difference turned out to be the location of the electrodes. The rats who couldn’t stop self-stimulating were the ones whose electrodes were activating dopaminergic pathways. These observations led to the idea that dopamine was the neural correlate of reward.

In the modern version of this study, researchers have used more refined techniques, such as optogenetics, to stimulate dopaminergic receptors in the striatum. In one study, mice were placed in a box with levers that could be activated by a nose poke (Kravitz et al., 2012). Touching one lever would turn on the laser and activate the cells. Touching the other did nothing. When the genetic label was targeted at dopaminergic receptors that were excitatory (D1 receptors of the direct, “go” pathway; see Chapter 8), the animals were much more likely to poke the lever (Figure 12.14). When the genetic label was targeted at dopamine receptors that were inhibitory (D2 receptor of the indirect, “no-go” pathway), the animals steered away from the lever.

Originally, neuroscientists thought that the stimulation caused a release of dopamine (or, in the optogenetic study, simulated the effects of dopamine release), and this event resulted in a pleasurable sensation. This hypothesis, however, turns out to be too simplistic, and understanding the effects of dopamine has become a major focus of interest in the neurosciences. A key challenge to the reward hypothesis came about when investigators recognized that the activation of dopaminergic neurons was not tied to the size of the reward per se, but was more closely related to the expectancy of reward (for a review, see Shultz, 1998). Dopaminergic neurons were especially active when a reward was unexpected. This observation led to a new view of the role of dopamine in reinforcement and decision making.

Dopamine and Prediction Error We know from experience that the value of an item can change. Your favorite fishing hole may no longer be a favorite with the fish. After a couple of unsuccessful visits, you update your value (now that spot is not your favorite, either) and look for a new fishing hole. How do we learn and update the values associated with different stimuli and actions? An updating process is essential, because the environment may change. Updating is also essential because our own preferences change over time. Think about that root beer float. Would you be so eager to drink one if you had just downed a couple of ice cream cones?

Wolfram Shultz and his colleagues at Cambridge University have conducted a series of revealing experiments using a simple Pavlovian conditioning task with monkeys (Chapter 9). The animals were trained such that a light, the conditioned stimulus (CS), was followed after a few seconds by an unconditioned stimulus (US), a sip of juice (Figure 12.15). To study the role of dopamine, Schultz recorded from dopaminergic cells in the ventral tegmental area (VTA). As expected, when the training procedure started, the cells showed a large burst of activity after the US was presented. Such a response could be viewed as representing the reward. When the CS-US events were repeatedly presented, however, two interesting things occurred. First, the dopamine response to the juice, the US, decreased over time. Second, the cells started to fire when the light, the CS, was presented. That is, the dopamine response gradually shifted from the US to the CS.

A reinforcement account of the reduced response to the US might emphasize that the value of the reward drops over time as the animal feels less hungry. Still, this hypothesis could not account for why the CS now triggers a dopamine response. It seems that the response here suggests that the CS has now become rewarding.

Schultz proposed a new hypothesis to account for the role of dopamine in reward-based learning. Rather than think of dopamine as representing the reward, he suggested that it should be viewed as a prediction error (PE), a signal that represents the difference between the obtained reward and the expected reward. First, consider the reduction in the dopaminergic response to the juice. On the first trial, the animal has not learned that the light is always followed by the juice. Thus, the animal does not expect to receive a reward following the light, but a reward is given. This event results in a positive prediction error (PPE), because the obtained reward is greater than the expected reward. With repeated presentation of the light–juice pairing, however, the animal comes to expect a reward when the light is presented. As the expected and obtained values become more similar, the size of the PPE is reduced, and thus, the dopaminergic response becomes attenuated.

Now, consider the increase in the dopaminergic response to the light. When the animal is sitting in the test apparatus between trials, it has no expectancy of reward. Initially, the animal does not associate the light with a reward. Thus, when it flashes, there is no PE: Expectancy is low, and the reward is associated with the juice, not the light. As the animal begins to associate the light with the juice, however, the onset of the light results in a PPE. The animal has no expectation (it is just hanging out), when it gets a stimulus associated with reward (yippee!). This PPE is represented by the dopaminergic response to the light.

The prediction error model has proven to be an important idea for thinking about how dopamine is related to both reinforcement and learning. In the previous example, we described positive prediction errors, the case where the obtained reward is greater than the expected reward. We can also consider situations with negative prediction errors, cases in which the obtained reward is less than the expected reward. This situation happens during a trial when the experimenter meanly withholds the juice after presenting the light. Now there is a dip in the response of the DA neuron around the time when the juice was expected (Figure 12.15C). This negative prediction error occurs because the animal is expecting the juice, but none is obtained. If the juice is repeatedly withheld, the size of both the increase in the dopaminergic response to the light and the decrease in the dopaminergic response to the absence of the juice are reduced. This situation corresponds to the phenomenon of extinction, where a response previously associated with a stimulus is no longer produced. With enough trials, the dopaminergic neurons show no change in baseline firing rates. The light is no longer reinforcing (so the PPE to the light is extinguished), and the absence of the juice is no longer a violation of an expectancy (so the negative prediction error, or NPE, when the juice was anticipated is also abolished).

As we have seen in this example, the dopaminergic response changes with learning. Indeed, scientists have recognized that the prediction error signal itself can be useful for reinforcement learning, serving as a teaching signal. As discussed earlier, models of decision making assume that events in the world (or internal states) have associated values. Juice is a valued commodity, especially to a thirsty monkey. Over time, the light also becomes a valued stimulus, signaling the upcoming reward. The PE signal can be used to update representations of value. Computationally, this process can be described as taking the current value representation and multiplying it by some weighted factor (gain) of the PE (Dayan & Niv, 2008). If the PE is positive, the net result is an increase in value. If the PE is negative, the net result is a decrease in value.

This elegant, yet simple model not only predicts how values are updated but also accounts for changes in the amount that is learned from one trial to the next. Early in training, the value of the light is low. The large PE that occurs when it is followed by the juice will lead to an increase in the value associated with the light. With repeated trials, though, the size of the PE decreases, so subsequent changes in the value of the light also will increase more slowly. This process, in which learning is initially rapid and then occurs in much smaller increments over time, is characteristic of almost all learning functions. Although this effect might occur for many reasons (e.g., the benefits of practice diminish over time), the impressive thing is that it is predicted by a simple model in which value representations are updated by a simple mechanism based on the difference between the predicted and obtained reward.

As we saw previously, many factors influence value, including magnitude, probability, and timing. Experiments have shown that the magnitude of a dopaminergic response varies with these factors and that this variation can be accounted for by the prediction error model. For example, the dopamine response to the US decreases when a CS more reliably predicts the reward (Fiorillo et al., 2003: Figure 12.16a), and it scales in the expected direction when the amount of reward is altered (Tobler et al., 2005). Another way to vary the PE is by lengthening the time between the CS and US. To take an extreme example, suppose that the CS occurs an hour before the US. Because of the long delay, it would be hard for an animal to learn the predictive value of CS, even if the US always followed. As shown in Figure 12.16b, the dopaminergic response to the CS is strongest at short delays and falls off with increasing delay (S. Kobayashi & Schultz, 2008). The opposite happens to the US with longer delays: The dopaminergic response to the US increases, indicating that the reward is more unexpected the further away in time it appears from the CS. Interestingly, these dopamine responses seem to mirror behavioral preferences: Large, probable, immediate rewards are preferred to smaller, less likely, and distant ones.

Reward and Punishment Not all options are rewarding: just consider your dog’s response after he has tried to nudge a porcupine out of a rotting tree. Talk about prediction error and learning by experience! Are positive and negative reinforcers treated by the same or different systems? Although it may seem like it, punishment is not the withholding of a reward. Whereas the absence of an expected reward is coded by negative prediction errors, punishment involves the experience of something aversive, like getting a shock or a nose full of porcupine quills. Aversive events are the opposite of rewarding events in that they are unpleasant, should be avoided, and have opposite motivational values.

In one important respect, however, reinforcement and punishment are similar. They are both motivationally salient, the kinds of events that draw our attention and engage control processes to influence behavior. The role of dopamine in aversive events has been difficult to pin down: Some studies show increases in dopamine activity, others find decreases, and some find both within the same study. Can these findings be reconciled?

The habenula, a structure located within the dorsal thalamus, is in a good position to represent emotional and motivational events because it receives inputs from the forebrain limbic regions and sends inhibitory projections to dopamine neurons in the substantia nigra pars compacta. Masayuki Matsumoto and Okihide Hikosaka (2007) recorded from neurons in the lateral habenula and dopaminergic neurons in the substantia nigra pars compacta while monkeys saccaded to a target that was either to the left or right of a fixation point. A saccade to one target was associated with a juice reward, and a saccade to the other target resulted in non-reinforcement. Habenula neurons became active when the saccade was to the no reward side and were suppressed if the saccade was to the reward side. DA neurons showed the opposite profile: They were excited by the reward-predicting targets and suppressed by the targets predicting no reward. Even weak electrical stimulation of the habenula elicited strong inhibition in DA neurons, suggesting that reward-related activity of the dopaminergic neurons may be regulated by input from the lateral habenula.

Value is in one sense relative. If given a 50–50 chance to win $100 or $10, we would be disappointed to get only $10. If the game were changed, however, so that we now stand to win either $10 or $1, we’re thrilled to get the $10. Habenula neurons show a similar context dependency. If two actions result in either juice or nothing, the habenula is active when the nothing choice is made. But if the two actions result in either nothing or an aversive puff of air to the eye, the habenula is active only when the animal makes the response that results in the airpuff. This context dependency is also seen in DA responses. In our two lottery examples, we might imagine that the expected reward in the first pairing is $55 (average of $100 and $10), whereas in the second pairing, it is only $5.50. The $10 outcome results in a positive prediction error in one case and a negative PE in the other. In sum, there are many computational similarities between how we respond to rewards and punishments, and this finding may reflect the interaction between the habenula and dopamine system.

In general, fMRI studies lack the spatial resolution to measure activity in small brainstem regions such as the VTA or lateral habenula. Nonetheless, researchers can ask similar questions about the similarity of neural regions in coding positive and negative outcomes. In one study, Ben Seymour and his colleagues (2007) paired different cues with possible financial outcomes that signaled a gain versus nothing, a loss versus nothing, or a gain versus a loss (Figure 12.17a). Study participants did not make choices in this experiment; they simply viewed the choices, and the computer determined the outcome. Positive and negative prediction errors of gains and losses were both correlated with activity in the ventral striatum, but the specific striatal region differed for the two conditions. Gains were encoded in more anterior regions and losses in the more posterior regions (Figure 12.17b). A region in the insula also responded to prediction error, but only when the choice resulted in a loss.

How does the release of dopamine actually result in learning? Although this question remains a hot topic of research, computational models have focused on how the PE signal might influence neural plasticity. One hypothesis is based on the idea of Hebbian learning (see Chapter 1) with a twist: “Neurons that fire together wire together, as long as they get a burst of dopamine.” Figure 12.18 illustrates a model of how positive and negative PE are thought to influence synapses in the striatum.

The PE story elegantly accounts for the role of dopaminergic cells in reinforcement and learning, but there remain viable alternative hypotheses. Kent Berridge (2007) argues that dopamine release is the result, not the cause, of learning. He points out a couple of problems with the notion that dopamine acts as a learning signal. First, mice that are genetically unable to synthesize dopamine can still learn (Cannon & Bseikri, 2004; Cannon & Palmiter, 2003). Second, genetically mutant mice with high dopamine levels do not learn any faster, nor do they maintain habits longer, than mice with normal levels of dopamine. Given these puzzles, Berridge suggests that dopamine neurons do not cause learning by encoding PE. Instead, they code the informational consequence of prediction and learning (generated elsewhere in the brain) and then do something with the information. He proposes that dopamine activity is indicative of the salience of a stimulus or an event.

Berridge describes a reward as made up of three dissociable components: wanting, learning, and liking. His view is that dopamine mediates only the “wanting” component. Dopamine activity indicates that something is worth paying attention to, and when these things are associated with reward, the dopamine activity reflects how desirable the object is. The distinction between wanting and liking may seem subtle, but it can have serious implications when we consider things like drug abuse. In one experiment, cocaine users were given a drug that lowered their dopamine levels (Leyton et al., 2005). In the lowered dopamine state, cues indicating the availability of the drug were rated as less desirable. When given the drug, however, the users’ feelings of euphoria and the rate of self-administration were unaffected. That is, with reduced dopamine, study participants still liked the drug in the same way (reinforcement was unchanged), even though they didn’t particularly want it.

It is, of course, reasonable to suppose that dopamine serves multiple functions. Indeed, neurophysiologists have described two classes of responses when recording from DA neurons in the brainstem (Matsumoto & Hikosaka, 2009b). One subset of dopamine neurons responded in terms of valence. These cells increase their firing rate to stimuli that are predictive of reward and decrease their firing rate to aversive stimuli (Figure 12.19a). A greater number of dopamine neurons, however, were excited by the increased likelihood of any reinforcement, independent of whether it was a reward or a punishment, and especially when it was unpredictable (Figure 12.19b). The first response class is similar to what would be expected of neurons coding prediction errors, the second to what would be expected of neurons coding salience or signaling things that require attention. Interestingly, the reward neurons were located more ventromedially in the substantia nigra and VTA, areas that project to the ventral striatum and are part of a network involving orbitofrontal cortex. In contrast, the neurons excited by salience were located more dorsolaterally in the substantia nigra, regions with projections to the dorsal striatum and a network of cortical areas associated with the control of action and orientation. We can see that when damage occurs within the dopamine system, or when downstream structures in the cortex are compromised, control problems are going to be reflected in behavioral changes related to motivation, learning, reward valuation, and emotion. These observations bring us back to how frontal lobe control systems are at work in both decision-making and goal-oriented behavior.

Once humans choose a goal, we have to figure out how to accomplish it. We usually make a plan in which we organize and prioritize our actions. Patients with prefrontal lesions, like W.R., often exhibit poor planning and prioritizing skills. Three components are essential for successfully developing and executing an action plan (Duncan, 1995). First, the person must identify the goal and develop subgoals. For instance, in preparing for an exam, a conscientious student develops an action plan like the one in Figure 12.20. This plan can be represented as a hierarchy of subgoals, each requiring actions to achieve the goal: Reading must be completed, lecture notes reviewed, and material integrated to identify themes and facts. Second, in choosing among goals and subgoals, consequences must be anticipated. Would the information be remembered better if the student sets aside 1 hour a day to study during the week preceding the exam, or would it be better to cram intensively the night before? Third, the student must determine what is required to achieve the subgoals. A place must be identified for study. The coffee supply must be adequately stocked. It is easy to see that these components are not entirely separate. Purchasing coffee, for example, can be an action and a goal.

When an action plan is viewed as a hierarchical representation, it is easy to see that failure to achieve a goal can happen in many ways. In the example illustrated in Figure 12.20, if reading is not completed, the student may lack knowledge essential for an exam. If a friend arrives unannounced the weekend before the exam, critical study time can be lost. Likewise, the failures of goal-oriented behavior in patients with prefrontal lesions can be traced to many potential sources. Problems can arise because of deficits in filtering irrelevant information, making it difficult to keep the eyes on the prize. Or the challenge may come in prioritizing information to help select the best way to achieve a particular goal or subgoal.

As discussed earlier, processing differences along the anterior–posterior gradient of PFC can be described in terms of the level of abstraction of action goals. As a clear demonstration of this hierarchy, consider an fMRI study in which participants completed a series of nested tasks (Badre & D’Esposito, 2007). The simplest task manipulated response competition by varying the number of possible finger responses to a series of colored squares presented one at a time (Figure 12.21a). In the color task, the response was based on the color of the square. The feature task added another layer of complexity because the response was based on texture and the colors indicated which response was associated with which texture. The third and fourth tasks added additional levels of complexity: The former required participants to use the color to determine the relevant dimension, and the fourth involved the manipulation of the stimulus–response mapping. Consistent with the hierarchical gradient hypothesis, more anterior regions of PFC were recruited as the task became more complex (Figure 12.21b). For the simplest task, varying response complexity activated premotor cortex. When the participant had to use color to select the appropriate dimension for the response, activation was also observed in more prefrontal cortex and extended into polar frontal cortex for the most complex task.

It might be supposed that, rather than reflect a hierarchy, the different activation patterns show that different subregions of PFC are required for things like response selection or rule specification. A key idea of hierarchy, however, is that processing deficits will be asymmetric. Individuals who fail at operations required for performance at the lower levels of a hierarchy will also fail when given more challenging tasks. In contrast, individuals who fail at tasks that require the highest levels should still be able to perform tasks that are dependent only on lower levels. This behavior is indeed what is observed in patients with PFC damage (Badre et al., 2009). If the lesions were restricted to the most anterior regions, the patients performed similar to controls on the first and second task conditions. Patients with more posterior lesions, those centered in premotor cortex, were impaired at all of the tasks.

A clever demonstration of the importance of the frontal lobes in this hierarchical evaluation process captured the real-world problems faced by patients with penetrating head injuries (Goel et al., 1997). The patients were asked to help plan the family budget of a couple who were having trouble living within their means. The patients understood the overriding need to identify places where savings could be achieved, and they could appreciate the need to budget. Their solutions did not always seem reasonable, however. For instance, instead of eliminating optional expenditures, one patient focused on the family’s rent. Noting that the $10,800 yearly expense for rent was by far the biggest expense in the family budget, he proposed that it be eliminated. When the experimenter pointed out that the family would need a place to live, the patient didn’t waver from his assessment and was quick with an answer: “Yes. Course I know a place that sells tents cheap. You can buy one of those.”

By focusing on the housing costs, the patient is perseverating, demonstrating inflexibility in his decision. The large price tag assigned to rent was a particularly salient piece of information, and the patient’s budgeting efforts were focused on the potential savings to be found here. From a strictly monetary perspective, this decision makes sense. But at a practical level, we realize the inappropriateness of this choice. Making wise decisions with complex matters, such as long-term financial goals, requires keeping an eye on the overall picture and not losing track of the forest because of the trees. To succeed in this kind of activity, we must monitor and evaluate the different subgoals. An essential feature of cognitive control is the ability to shift our focus from one subgoal to another. Complex actions require that we maintain our current goal, focusing on the information that is relevant to achieving that goal, ignore irrelevant information, and, when appropriate, shift from one subgoal to another in a coordinated manner.

Goal-oriented behavior requires people to select task-relevant information and filter out task-irrelevant information. Here selection refers to the ability to focus attention on perceptual features or information in memory. This selection process is a cardinal feature of tasks associated with the lateral prefrontal cortex, highlighting its role in working memory and attention.

Suppose that you are telling a friend about walking across the Golden Gate Bridge during a recent trip to San Francisco (Figure 12.22). The conversation will have activated semantic information from your long-term memory about the location, shape, and color of the bridge, as well as episodic information related to your trip. These representations constitute the contents of working memory. If your friend then asks you about the color of the bridge, you must be able to focus on your memory of the color of the bridge. This example demonstrates that working memory is more than the passive sustaining of representations. It also requires an attentional component in which the participant’s goals modify the salience of different sources of information. To capture this idea, the PFC has been conceptualized as a dynamic filtering mechanism (Shimamura, 2000). Reciprocal projections between PFC and posterior cortex provide a way for goals, represented in PFC, to maintain task-relevant information that requires long-term knowledge stored in posterior cortex. As the goals shift—say, from recalling the walk across the bridge to remembering the color of the bridge—the filtering process will make salient links to representations associated with the color.

The filtering hypothesis offers a way to appreciate the role of the frontal lobe in tasks where memory demands are minimal. Frontal lobe patients display heightened interference on the Stroop task, in which participants are shown a list of colored words and the words spell color names such as red, green, or blue. In the congruent condition, the colors of the words correspond to their names; in the incongruent condition, the word names and colors do not correspond (see Figure 3.5). With years of reading experience, we have a strong urge to read words even when the task requires us to ignore them in favor of color. Thus everyone is slower in responding to incongruent stimuli in comparison with congruent stimuli. This difference is even greater in patients with frontal lobe lesions.

The contribution of prefrontal cortex to selection is evident in a series of elegant experiments conducted by Sharon Thompson-Schill (Thompson-Schill et al., 1997, 1998). In early PET studies on language, experimenters found that when participants were given a noun and had to generate a semantically associated word, a prominent increase in activation was observed in the inferior frontal gyrus of the left hemisphere. Thompson-Schill hypothesized that this prefrontal activation reflected filtering of the transient representations (the semantic associates of the target item) as they were being retrieved from long-term memory in the posterior cortex. To test this hypothesis, the researchers conducted an fMRI study in which they varied the demands on a filtering process during a verb generation task (Figure 12.23). In the low-filtering condition, each noun was associated with a single verb. For example, when asked to name the action that goes with scissors, almost everyone will respond “cut,” and thus, there is no need to filter out competing, alternative responses. In the high-filtering condition, however, each noun had several associates. For example, for the noun rope, multiple answers are reasonable, including the verbs tie, lasso, and twirl. Here, a filtering process is required to ensure that one answer is selected. Note that, in both conditions, the demands on semantic memory are similar. The participant must comprehend the target noun and retrieve semantic information associated with that noun. If this region is involved in the active retrieval of goal-related information, however, then activation should be greater in the high-filtering condition. The experiment results supported this prediction.

Consider the following puzzle: Your task is to fix a candle to the wall of a room and light it. You have been supplied with a candle, a box of matches, and some thumbtacks. Are you up for the challenge? Go.

You probably solved this rather quickly. Simply take a thumbtack, stick it through the candle and into the wall, and then light the candle. Not so fast. We forgot to tell you that the diameter of the candle is much thicker than the length of the thumbtack. Take another shot.

Stumped? Don’t be discouraged—thousands of students have been mystified by this brainteaser since Rainer Dunker introduced it in his monograph on problem solving in 1945 (cited in Wickelgren, 1974). Here’s a hint: Suppose that there is only one match, and it sits on the table outside the matchbox. Now give it another go.

When the problem is presented in this format, many people experience an “aha” moment. They suddenly realize that the matchbox can serve more than one purpose. In addition to providing a striker for the matches, it could be used as a crude candlestick. Tack the box to the wall with the thumbtacks, light the candle and let it drip into the box, and then set the candle in the goo so that when the drippings cool, the candle will be secure in an upright position.

These problems are challenging because stimuli trigger the retrieval of associations that we have made previously. Thus, in developing an action plan, we tend to think narrowly about the possible uses of an object—a phenomenon that psychologists refer to as functional ﬁxedness. Functional fixedness might be seen as an undesired consequence of having evolved the kind of rapid-response selection ability associated with the prefrontal cortex. As we have seen in the work on semantic generation, the lateral prefrontal cortex facilitates the selection of viable responses from a set of automatically activated long-term representations. With the matchbox, we immediately think of its common use—to light matches—and then mull over how those thumbtacks can be applied to the candle. By emptying the box of matches, we might realize new possibilities; but even here, many people continue to be unable to see novel uses because of the strong association between the stimulus and an action. Chris Frith (2000) has referred to the selection process of lateral prefrontal cortex as “sculpting the response space.”

Ready for your next brainteaser? Here is a false arithmetic statement in Roman numerals, represented by matchsticks:

Problem 1: VI = VII + I

Provide a correct solution by moving only one stick. Not too hard. Moving one of the Is from the VII to the VI renders the correct statement, VII = VI + I.

Now try a problem that, with the one-move rule, is much more difficult:

Problem 2: VI = VI + VI

Stuck again? Moving a matchstick from one of the VIs on the right side of the equation to the left won’t do it. Nor will turning a VI into a IV. The answer here requires an unusual transformation of one of the operators and a kind of arithmetic statement that we rarely encounter: VI = VI = VI.

On the basis of the selection hypothesis, Carlo Reverberi and his colleagues at the University of Milan (2005) made an unusual prediction. They proposed that patients with lateral prefrontal cortex lesions would actually do better on Problem 2 than would healthy control participants. This prediction was based on the idea that an impaired selection process would make it easier for the patients to represent atypical actions. Indeed, this is exactly what they found. The superior performance of the patients was especially striking, given that these individuals were worse than the controls when presented with equations like those in Problem 1 or equations that required standard operator transformations (e.g., V = III – II, in which the equal and minus signs are swapped by the movement of one matchstick). Here the patients’ impairment became greater as the number of possible moves increased, consistent with the idea that the lateral prefrontal cortex is especially critical when the response space must be narrowed. But for equations like Problem 2, the “sculpting” process of prefrontal cortex led the controls to focus on the numbers or simple changes in the operators (e.g., turn the plus sign into a minus).

FIGURE 1 Patients with lateral prefrontal lesions do better than healthy control participants on a problem-solving task that requires unusual solutions.
For the easy and hard conditions, the solution requires moving a matchstick from one side of the equation to the other to transform a numeral or the operators. For the atypical condition, the solution requires rotating a matchstick to create a three-part equality.

These results are especially compelling when we consider that neuropsychological studies rarely involve tasks in which a patient group performs better than a control group. By thinking deeply (and outside the box) about the implications of a theory regarding prefrontal function, the researchers were able to recognize that processes that confer a functional advantage in most situations—rapidly selecting task-relevant responses—may not be optimal in certain situations.

One of those situations may be when we are young, leading some evolutionary theorists to revisit the question of why the frontal lobes mature late. The traditional view has been that the delayed maturation of the frontal lobes is an example of ontogeny following phylogeny: A late addition in evolution means late development. Thus the frontal lobes develop late in the child because the expansion of the frontal lobes is a relatively late adaptation. This point of view leads to a focus on the costs of not having a mature frontal lobe. Children have a hard time engaging in delayed gratification, maintaining focus, and inhibiting behavior.

Yet, we can also ask if there are advantages to this “delay” in development. One hypothesis is that an immature frontal lobe might make a person more open-minded, perhaps because they don’t have strong response associations to environmental cues or have well-established value representations. Such properties are good for learning. The child does not respond to a situation in a predictable manner, but rather is open to recognizing new contingencies. Linda Wilbrecht and her colleagues (Johnson et al., 2011) looked at this idea in mice. They trained juvenile and adult mice to discriminate between four odors, learning that one of the odors was associated with a reward. After a number of trials, the odor– reward pairs were changed. The juvenile mice learned more quickly than the adult mice, a result reminiscent of the novel problem-solving abilities of patients with frontal lobe damage.

As can be seen in Figure 12.23c, the demanding version of the generation task also was associated with an increased BOLD response in the left temporal lobe. As we learned in Chapter 9, this area is hypothesized to be an important component of semantic memory. Indeed, the results of a follow-up study support this hypothesis (Thompson-Schill et al., 1999). Participants were trained to make two types of generation responses, one based on naming an action associated with the noun and another based on naming the color associated with the noun. The initial scanning run revealed a replication of the prefrontal and temporal cortical engagement during the generation tasks, demonstrating that the same inferior frontal region was recruited for both types of semantic associations.

Of special interest, however, was what happened in later scanning runs. The list of nouns was repeated. In one condition, participants performed the same generation task as for the first run; in the other, they were required to perform the alternative generation task of naming a color. This manipulation led to an interesting dissociation between the BOLD response in the prefrontal and temporal cortices. Prefrontal activation increased in scanning runs in which the generation requirements changed. Selection and filtering likely would be high under such conditions. A different pattern was seen in the temporal lobe. Here the activation decreased on the second run for both the same and the different generation conditions. Such decreases with repetition have been seen in many imaging studies of priming (see Chapter 9). The fact that the decrease was observed even when the generation requirements changed is consistent with the idea that semantic attributes, whether relevant or irrelevant to the task at hand, are automatically activated upon presentation of the nouns. The prefrontal cortex applies a dynamic filter to help retrieve and select information that is relevant to the current task requirements.

The loss of dynamic filtering captures an essential feature of prefrontal damage. The patients’ basic cognitive capabilities are generally spared, their intelligence shows little evidence of change, and they can perform normally on many tests of psychological function. In an environment where multiple sources of information compete for attention, however, these patients are in a particularly vulnerable condition: They have difficulty maintaining their focus on a goal.

To study this aspect of cognitive control, researchers have developed experiments to study task switching. One example is shown in Figure 12.24. On each trial, a letter–digit pair is presented. The task goal is switched every two trials, alternating between trials in which the participant is required to name the digit and trials in which the participant is required to name the letter. Thus, in the first trial for each pair, the task goal switches (e.g., changes from naming the letter to naming the digit). In the second trial, however, the task goal remains the same. The time required to change from one goal to the other, the switching cost, is measured by the difference in response time on these two types of trials. A second important variable in this experiment is how the task goal is specified. Because the trials alternate consistently, participants can keep track of their place; but doing so increases their processing requirements. To avoid complicating the processing, the task goal is cued by an external cue. In one condition, this cue is indicated by the background color (Figure 12.24a). In the other condition, a visual word cue is used (Figure 12.24b).

As Figure 12.24c shows, the type of cue turned out to be critical. When a visual word cue was used to specify the task goal, patients with lateral prefrontal lesions performed similarly to the matched control participants. When a color cue was used, however, the patients were slow on the switch trials. This dissociation reinforces the idea that the prefrontal cortex is important for coordinating goal-oriented behavior. Moreover, this form of control is needed especially when the goal must be retrieved from memory. With the color cue, the patients must remember the associations between the colors and the tasks (e.g., red with digit naming). The word cues do not require this step of referring to memory.

Dynamic filtering is a form of goal-based control. In the Golden Gate Bridge and semantic association examples, this goal-based control was seen as facilitating the retrieval of some information from long-term memory. In the Stroop task, goal-based control facilitates online processing in a similar way. The task goal requires that we attend to one visual dimension (the color of the ink) while ignoring another visual dimension (the word). A focus of considerable research has been to understand the neural mechanisms through which goal-based control is achieved. This problem is especially challenging because it requires understanding interactions between brain regions that are considerably far apart. Jon Driver and his colleagues at University College London provided one remarkable example of how the brain coordinates activity across different neural regions. It required a special TMS device that could be used in the MRI scanner (Ruff et al., 2006).

The researchers set out to investigate how disruption of the frontal cortex affected processing in posterior cortex. They targeted the frontal eye field (FEF) region in prefrontal cortex (see Chapter 7), a region that plays a role in spatial attention by working in tandem with the superior colliculus to control eye movements. Attention requires a balance between maintaining the current focus of attention and orienting to novel salient stimuli. The FEF is critical for overcoming the tendency to look at novel stimuli. Thus the researchers were interested in how TMS over the FEF would influence retinotopic maps in early visual areas. They repeated the well-tested fMRI method of mapping visual areas (see Chapter 5) but interspersed the visual stimuli with short bursts of TMS pulses under the assumption that these pulses would cause transient disruption of the FEF (Figure 12.25).

Compared to a condition in which the TMS was directed to a control site, the BOLD response was attenuated in those portions of the visual areas that represented central vision. More surprising, activation was increased in the regions that represented peripheral vision. These results suggest that, without the goal-based influence from the FEF to maintain focus, perceptual signals arising in the fovea were attenuated while those from the periphery become more salient. This prediction was confirmed in a follow-up behavioral study. Participants judged flashes of light in the periphery as brighter following FEF TMS when compared to TMS over a control site.

Goal Representation and the Inhibition and Enhancement of Working Memory Representations

Goal-based control could influence the contents of information processing in at least two distinct ways. One is to accentuate the attended information. For example, when we attend to a location, our sensitivity to detect a stimulus at that location is enhanced. Alternatively, we can selectively attend by excluding information from other locations. Similarly, when multiple sources of information come from the same location, we might selectively enhance the task-relevant information (color in the Stroop test) or inhibit the irrelevant information (the word in the Stroop test). In behavioral tasks, it is often difficult to distinguish between facilitatory and inhibitory modes of control. Moreover, as seen in times of budgetary crises, the hypotheses are not mutually exclusive. If we have fixed resources, allocating resources to one thing places a limit on what is available for others; thus, the form of goal-based control may vary as a function of task demands.

No doubt your cognitive neuroscience lectures are riveting and have your full attention. We know that in your other less interesting classes, you are listening to your professor while also shifting your attention, texting friends, and surfing the Web. In common speech, you are multitasking. But when multitasking, are we really doing two goal-related activities at once, or are we simply quickly switching between two tasks?

A favorite way to study this behavior is to combine a visual– manual task (e.g., press one of two buttons to indicate the position of a stimulus) and an auditory–vocal task (e.g., hear two arbitrary sounds and say “Tay” to one and “Koo” to the other). People are tested on each task alone or with a visual stimulus and auditory stimulus presented simultaneously (a dual task). At first, participants do much worse in the dual-task condition, but after 5 days or so of training, they can do the two tasks simultaneously with little to no interference (Hazeltine et al., 2002; Schumacher et al., 2001). Thus, with practice, people get quite good at multitasking. How do we achieve this?

There are two hypotheses about how we become proficient multitaskers. One is that we learn to segregate the two tasks, doing each in parallel. The other is that we become proficient in switching from one task to the other. Frank Tong and colleagues (Dux et al., 2009) performed an innovative fMRI study, scanning participants repeatedly over a 2-week period as they practiced performing two tasks simultaneously (visual–manual and auditory–vocal). As expected, the participants showed a large reduction in dual-task cost (faster reaction time with no loss in accuracy) after training. They then looked at the connectivity patterns, focusing on the inferior frontal cortex based on prior imaging results showing activation in this region in task-switching studies (see the Konishi study discussed earlier in this chapter). This region showed a significant reduction in activity with training, consistent with the idea that the tasks are becoming segregated. The functional connectivity data, however, revealed a different pattern. Inferior frontal cortex remained strongly connected with both auditory cortex (AC) and visual cortex (VC) as well as with two regions of motor cortex, one associated with manual responses and the other with vocal responses (Figure 1). In addition, with training, the peak of the frontal response came earlier and was of shorter duration: evidence that the participants were becoming more efficient in switching. This study suggests that we aren’t really multitasking, but quickly alternating between tasks.

FIGURE 1 Functional connectivity of control network with practice in multitasking.
(a) Functional connectivity between prefrontal cortex and perceptual-motor areas for a visual-manual task (green) or auditory-vocal task (red). (b) Connectivity strength before and after two weeks of multitask training. Connectivity was strong in a task-specific manner and showed little change, even though the participants became extremely proficient in performing the two tasks simultaneously. (c) Activity in prefrontal cortex remained high, and shifted to an earlier latency after training in the dual-task condition, suggesting persistent cognitive control.

Evidence for a loss of inhibitory control with frontal lobe dysfunction comes from electrophysiological studies. Robert Knight and Marcia Grabowecky (1995) recorded the evoked potentials in groups of patients with localized neurological disorders. In the simplest experiment, participants were presented with tones, and no response was required. As might be expected, the evoked responses were attenuated in patients with lesions in the temporoparietal cortex in comparison to control participants. This difference was apparent about 30 ms after stimulus onset, the time when stimuli would be expected to reach the primary auditory cortex. The attenuation presumably reflects tissue loss in the region that generates the evoked signal. A more curious aspect is shown in Figure 12.26. Patients with frontal lobe lesions have enhanced evoked responses. This enhancement was not seen in the evoked responses at subcortical levels. The effect did not reflect a generalized increase in sensory responsiveness, but was limited to the cortex.

The failure to inhibit irrelevant information was more apparent when participants in this study were instructed to attend to auditory signals in one ear and ignore similar sounds in the opposite ear, when signals in the attended ear varied between blocks (see Figure 12.26b). In this way, an assessment can be made of the evoked response to identical stimuli under different attentional sets (e.g., response to left-ear sounds when they are attended or ignored). With healthy participants, these responses diverge at about 100 ms; the evoked response to the attended signal becomes greater. This difference is absent in patients with prefrontal lesions, especially for stimuli presented to the ear contralateral to the lesion (e.g., left ear for a patient with a lesion in right hemisphere prefrontal cortex). What happens is that the unattended stimulus produces a heightened response. This result is consistent with the hypothesis that the frontal lobes modulate the salience of perceptual signals by inhibiting unattended information.

In the study just described, we can see inhibition operating to minimize the impact of irrelevant perceptual information. This same mechanism can be applied to memory tasks for which information must be internally maintained. Consider the monkey attempting to perform a delayed-response task (see Figure 12.2). The monkey views the target being placed in one of the food wells and then the blind is closed during the delay period. The monkey’s mind does not just shut down; the animal sees and hears the blind being drawn, looks about the room during the delay interval, and perhaps contemplates its hunger. All such intervening events can distract the animal and cause it to lose track of which location is baited. To succeed in finding the food, it must ignore the distractions and sustain the representation of its forthcoming response. We have all experienced failures in similar situations. A friend gives us her telephone number, but we forget it. The problem is not a failure to encode the number. Something else captures our attention, and we fail to block out the distraction. This point is underscored by the finding that primates with prefrontal lesions perform better on delayed-response tasks when the room is darkened during the delay (Malmo, 1942) or when they are given drugs that decrease distractibility.

The preceding discussion emphasizes how goal-based control might be achieved by the inhibition of task-irrelevant information. Mark D’Esposito and his colleagues (Druzgal & D’Esposito, 2003) used fMRI to further explore interactions between prefrontal cortex and posterior cortex. In a series of experiments, they exploited the fact that regions in the inferior temporal lobe are preferentially activated by face and place stimuli—the so-called FFA (fusiform face area) and PPA (parahippocampal place area), respectively (Chapter 6). The researchers asked whether activation in these regions is modulated when people are given the task goal to remember either faces or places for a subsequent memory test (Figure 12.27). At the start of each trial, an instruction cue indicated the current task. Then a set of four pictures was presented; it included two faces and two scenes. As expected, a subsequent memory test verified that the participants selectively attended to the relevant dimension.

More interesting was the finding that activation in the FFA and PPA was modulated in different ways by the instruction cues (Figure 12.27b), showing both enhancement and suppression effects. Compared to the passive viewing condition (control), the response in the FFA of the right hemisphere was greater when the participants were instructed to remember the faces and lower when the participants were instructed to remember the scenes. The reverse pattern was evident in the PPA, and here the effect was seen in both hemispheres. This study reveals that the task goal, specified by the instruction, can modulate perceptual processing by either amplifying task-relevant information or inhibiting task-irrelevant information.

In an interesting extension, the experiment was repeated, but this time the participants were older, neurologically healthy individuals (Gazzaley et al., 2005b). Unlike college-age participants, the older participants showed only an enhancement effect; they did not show the suppression effect in either FFA or PPA when results were compared to the passive viewing condition (Figure 12.27c). These findings are intriguing for two reasons. First, they suggest that enhancement (i.e., amplification) and suppression (i.e., inhibition) involve different neural mechanisms and that inhibition is more sensitive to the effects of aging. Second, given that aging is thought to disproportionately affect prefrontal function, perhaps inhibitory goal-based control is more dependent on prefrontal cortex than are the attentional mechanisms that underlie the amplification of task-relevant information.

The work described in the previous section reveals that the task goal, specified by the instruction, can modulate perceptual processing by either amplifying task-relevant information or inhibiting task-irrelevant information. The data do not reveal, however, if this modulation is the result of prefrontal activation. To explore this question, researchers have applied TMS over prefrontal cortex and then asked how this perturbation affects processing in posterior perceptual areas. In one study, TMS was applied over inferior frontal cortex while participants were instructed to attend to either the color or motion of a visual stimulus (Zanto et al., 2011). Not only was performance poorer after TMS, but the difference between the P100 to the attended and ignored stimuli was reduced (Figure 12.28). This reduction occurred because after TMS, the P100 was larger for the ignored stimuli. In another study (Higo et al., 2011), participants received either low-frequency repetitive TMS or sham stimulation over prefrontal cortex. They next entered an fMRI machine where measurements were taken while they attended to places, faces, or body parts. TMS attenuated the modulation of category-specific responses in posterior cortex due to the participants’ attentional set. Moreover, the results indicated that the effects of frontal TMS primarily disrupted the participants’ ability to ignore irrelevant stimuli but had little effect on their ability to attend to relevant stimuli, a dissociation similar to that described above for older participants.

In a related study, Eva Feredoes and Jon Driver (Feredoes et al., 2011) combined TMS and fMRI during a working memory task, targeting dorsal prefrontal cortex. Unlike what has been reported for inferior frontal stimulation, TMS over dorsal PFC led to an increased BOLD response in task-relevant areas (e.g., increased FFA response when responding to faces) when distractors were present.

Let’s take a moment to put together these different results. TMS over frontal cortex led to a change in processing within posterior cortex, consistent with the general idea that goal-based representations in prefrontal cortex are used to modulate how perceptual information is selectively filtered. Moreover, the results might be taken to suggest that inferior frontal cortex is important for inhibiting task-irrelevant information, and dorsal frontal cortex is important for enhancing task-relevant information. This hypothesis, however, has a problem: It requires assuming that the effect of TMS in these studies was to disrupt processing when applied over inferior frontal cortex and to enhance processing when applied over dorsal frontal cortex. Although this effect is possible, especially since the TMS protocols were not identical in the different studies, it is also possible that disrupting one part of prefrontal cortex with TMS produces changes in other prefrontal regions. Perhaps TMS over dorsal PFC has a side effect of improving processing within inferior PFC. If this hypothesis were correct, then the task-relevant enhancement observed in the Feredoes study is showing a picture similar to that in the other studies. That is, TMS over inferior frontal cortex directly disrupts goal-based selection, while TMS over dorsal frontal cortex produces an indirect benefit in goal-based selection by increasing reliance on inferior frontal cortex. It may be that, neurally, competitive processes operate across our frontal gradients (e.g., dorsal–ventral). At present, we can only speculate on such hypotheses.

In most situations, our goals specify the information that we want to highlight. When scanning a crowd at the airport for our parents, we may activate perceptual units that correspond to their salient features, such as dad’s shaved head and mom’s big red hair. When hiking to our favorite fishing spot, we may look for familiar landmarks. We also appear to be capable of actively preventing some information from entering working memory. This form of suppression is different, however, from that required when we want to avoid being distracted by irrelevant information. Rather, goal-based control in this context might be useful for preventing undesirable information from gaining access to long-term memory. This kind of control is, in a sense, reminiscent of the idea that Sigmund Freud had when he developed his theory of the unconscious mind, suggesting that we repress unwanted thoughts and desires.

Michael Anderson at the Cognition and Brain Sciences unit in Cambridge, England, offers a 21st-century perspective on the question of active repression using a clever experimental design. Participants initially learned a set of word–pair associations such as steam–train. Later they were given the first word as a cue and told: Do not think of the associated word! This is, of course, a hard thing to do. If you are told not to think about pink elephants, probably the first thing that pops into your mind is a pink elephant. Yet, in subsequent memory tests, the participants were poorer at remembering the word pairs that they were instructed to keep out of mind.

Anderson hypothesizes that this active repression involves another form of goal-based control (Anderson and Levy, 2009). Using fMRI, he showed that activation in prefrontal cortex was greater when participants were presented with a single word cue and instructed not to think about its associate (e.g., if cued with steam, do not think of train) compared to a condition in which they were instructed to remember the word pair. This finding is consistent with our phenomenal experience—it requires effort not to think about something. Paralleling this enhanced prefrontal activation was a reduction in hippocampal activation during the active suppression of memory. The behavioral consequences of this interaction were evident in a subsequent memory test in which participants’ recall of the associated item was weakest when prefrontal activity was greatest. Importantly, control experiments, using indirect methods to assess performance, excluded the possibility that participants were simply pretending not to remember the word pairs. From this work we can conclude that, in addition to inhibiting irrelevant information, goal-based control processes associated with the prefrontal cortex can either amplify or inhibit relevant information, depending on the behavioral goals.

Inhibitory control can take many forms. We have seen that failures of inhibition lead to greater distractibility, a hallmark of prefrontal dysfunction. As described in the preceding section, inhibition may also be deployed to control access to long-term memory. Inhibition is useful for cognitive control in another circumstance: when we are about to take an action and something makes us change our mind. For instance, you are about to swing at a baseball when you see it is curving out of your reach. You suddenly realize that your selected response is not appropriate. You can’t reach the ball and will end up with a strike, so you abort the action before fully completing it. This form of inhibition—the cancellation of a planned action—is actually quite common even for those of us who never play baseball. At a party, we often find ourselves ready to jump in with a scintillating comment, only to find that the loudmouth bore (who apparently is not so good at inhibition) has once again commandeered the conversation. Politely, we hold back, waiting for another opportunity.

Inhibition of this form can be seen as the opposite of selecting an action. Are the neural mechanisms the same? That is, to inhibit an action, do we deselect that action by generating some sort of negative image of the brain activation? Even if we could do this, it might not be enough to inhibit the unwanted action. The commands to produce the planned action have already been sent to the motor system, and simply deactivating the plan would not be enough to stop an initiated movement.

This form of inhibitory control has been studied with the stop-signal task. In the standard form of this experiment, participants are tested in a reaction time task in which they have to choose between two alternatives. For example, if an arrow points to the left, press one button; if to the right, another button. The twist is that on some of the trials, a signal pops up indicating that the response should be aborted. This stop signal might be a change in color or the presentation of a sound. The time between the onset of the initial stimulus and the stop signal can be adjusted, creating a situation in which the participant sometimes succeeds in aborting the planned response and sometimes fails to abort the response. Three conditions result: (a) trials without a stop signal (go trials), (b) trials in which the person is able to stop (successful stop trials), and (c) trials in which the person fails to stop (failed stop trials; Figure 12.29).

Adam Aron at the University of California, San Diego, has employed a multimethod approach to establish the neural network underlying this form of cognitive control (Aron & Poldrack, 2006). Patients with lesions of the frontal lobe are slow to abort a planned response. This impairment appears to be specific to lesions of the inferior frontal gyrus on the right side, since the deficit is not present in patients with left frontal lesions or in patients with damage restricted to more dorsal parts of the right prefrontal cortex. The association of the right prefrontal cortex with this form of inhibitory control is also supported by fMRI data obtained in young adults. Here the BOLD response can be plotted for each of the three trial types (Figure 12.29b). Successful stop trials and failed stop trials both produce a strong response in the right inferior frontal gyrus. In contrast, this area is silent on go trials. The fact that the BOLD signal is very similar for both types of stop trials suggests that an inhibitory process is recruited in both situations, even though the control signal to abort the response is effective on only some trials.

Why might this be? The BOLD response in motor cortex is revealing. Here we see strong activation on both go trials and failed stop trials. The activation in motor cortex on the failed stop trials is already high, however, when the initial stimulus is presented (time 5 0). Note that the participants are under a lot of pressure in these experiments to go as fast as possible. This pre-stimulus activation likely reflects a high state of anticipation. Even though the right prefrontal cortex generates a stop command, the initial level of activation in motor cortex has led to a fast response, and the person is unable to abort the movement. For an antsy baseball player initially fooled by a curveball, it’s strike three.

The right inferior frontal gyrus pattern of activation was also present in the subthalamic nucleus (STN) of the basal ganglia. As we saw in Chapter 8, the basal ganglia are implicated in response initiation. Within this subcortical system, the STN provides a strong excitatory signal to the globus pallidus, helping maintain inhibition of the cortex. The stop-signal work suggests how this inhibition might be recruited within the context of cognitive control. Activation of the right prefrontal cortex generates the command to abort a response, and this command is carried out by recruiting the STN. This hypothesis led Aron and his colleagues to predict the existence of an anatomical connection between the right prefrontal cortex and the STN (Aron et al., 2007). Using diffusion tensor imaging (DTI), the researchers confirmed this prediction (Figure 12.29c). With an elegant design combining behavioral and fMRI results, they uncovered an anatomical pathway that had never been described in the literature. This anatomical pathway includes the pre-supplementary motor area, a part of the medial frontal cortex. As we will see shortly, this region is activated in functional imaging studies when response conflict occurs—something that obviously happens in the stop-signal task.

Recall from Chapter 8 that deep-brain stimulation (DBS) in the STN is used to treat Parkinson’s disease, improving the patient’s ability to initiate movement. Michael Frank and colleagues of Brown University (Frank et al., 2007) suggest that this procedure comes with a cost, one in which the person may become too impulsive because the stimulation disrupts a system for inhibitory control. To show this, they compared a group of control participants to Parkinson’s disease patients who had DBS implants. The participants were initially trained with a limited set of stimulus pairs. Within each pair, the stimuli had a particular probability of winning a reward. For instance, as seen in Figure 12.30, the first symbol is associated with a reward 80 % of the time and the second symbol with a reward 20 % of the time. Thus, some stimuli had a high probability, others a low probability. During the test trials, the experimenter introduced new combinations of the stimuli. In this way, some of the new pairs presented little conflict, because one item was much more likely than the other to lead to reward (e.g., if the pair included a stimulus with a win probability of 80 % and a stimulus with a 30 % win probability). Other pairs entailed high conflict, either because both items were associated with high reward probabilities (70 % and 60 %), or both were associated with low reward probabilities (30 % and 20 %). As expected, control participants were faster to respond on the low-conflict trials. The same pattern was observed for the patients with Parkinson’s disease when tested with the stimulator turned off, even though their reaction times were slower. When the stimulator was turned on, the patients responded faster, but they were no longer sensitive to the conflict: They responded faster to the high-conflict trials, especially when they made the wrong choice. Thus, although DBS can help alleviate the motor symptoms of Parkinson’s disease, it comes at the cost of being too impulsive.

Ticket to ride, white line highway...

Pay your toll, sell your soul

Pound for pound costs more than gold

The longer you stay, the more you pay

—“White Lines (Don’t Do It),” by Melle Mel and Grandmaster Flash (Sugar Hill Records, 1983)

A hallmark of drug addiction is the sense of a loss of control. The alcoholic will throw away a week’s paycheck buying rounds at the bar, momentarily ignoring that the rent is due next week and her credit card bill has gone unpaid for months. The addict feels like a slave to his drug of choice, forsaking all responsibilities when in need of his next fix. Given the physical, mental, and financial costs, both for the afflicted individual and for society in general, the study of drug abuse has been a top priority in the neuroscience community.

This research can be especially challenging because it is usually not possible to randomly assign participants to different experimental groups. We cannot designate certain individuals to become drug addicts, others to be abstinent. Rather, the assignments are based on individual histories, so it is often difficult to control for various secondary factors associated with drug addiction. Even so, studies on addiction have begun to offer new insights into the changes that occur in cognitive control with substance abuse.

Hugh Garavan and colleagues at Trinity College in Dublin, Ireland, have conducted a series of studies to look at this question in cocaine users (Kaufman et al., 2003). One of their tasks involves a simple test of inhibitory control. Participants view a stream of stimuli that alternate between two letters and quickly press a button (go trial) with the presentation of each letter (see Figure 1a). In rare instances, the same letter is repeated on successive trials. For these trials, they are instructed to withhold their response (no-go trials).

Chronic cocaine users, none of whom had used cocaine for 18 hours before testing, were more likely to respond on no-go trials than were matched controls, and they showed lower activation in the medial frontal cortex when they produced these erroneous responses, indicating that they had difficulty monitoring their performance (Figure 1b). What’s more, even when they succeeded in inhibiting the response, the medial frontal cortex response was lower in the cocaine users. Interestingly, they showed a stronger response in other brain areas, such as the cerebellum, that may reflect a compensatory process. Thus, even when the addicts were not under the influence of cocaine, changes in their cognitive control network persisted.

In subsequent work, Garavan and his colleagues have found that this impairment in inhibitory control is especially pronounced when working memory demands are high. This interaction of working memory and cognitive control may help us understand, at least in part, why addiction can be so hard to break. When addicts experience the craving for their drug of choice, working memory is taxed: Not only must these individuals attend to whatever they are currently doing, but the goal of obtaining the drug is highly activated and has been shown to produce strong responses to environmental cues associated with that drug. Under such conditions, the ability to monitor and inhibit behavior is compromised, increasing the likelihood of relapse.

This vicious cycle can interfere with recovery, even in individuals who are highly motivated to escape their drug addiction. By appreciating how motivation must be coupled with the appropriate mechanisms of cognitive control, researchers can consider therapeutic options that might facilitate the process. For example, prescription drugs that restore medial frontal activity to normal levels might prove useful in treating addiction.

FIGURE 1 Reduced inhibitory control in chronic cocaine users.
(a) Participants view a stream of letters and press a key whenever the letters alternate between X and Y (go trials). In a small percentage of the trials, the same letter appears in successive displays, and here the participants must withhold their responses (no go). (b) Activation in the medial frontal cortex is lower in the cocaine users on all no-go trials—whether inhibition of the response succeeds or fails.

Tim Shallice and Donald Norman developed the psychological model in Figure 12.31 to account for goal-oriented behavior. Like the concepts developed in Chapter 8, this model conceptualizes the selection of an action as a competitive process. At the heart of the model is the notion of schema control units, or representations of responses (a term used in a generic sense here). These schemas can correspond to explicit movements or to the activation of long-term representations that lead to purposeful behaviors. They are activated by either perceptual stimuli or another recently activated schema. For example, hearing your phone ring may activate the motor schema to answer it, or seeing a word printed on paper can activate a schema for an articulatory gesture. The activated schema of reading the word may in turn activate the semantic meaning and associated representations.

Schema control units receive input from many sources. Norman and Shallice emphasized perceptual inputs and their link to these control units, but it is the strength of the connections between the two that reflects the effects of learning. If we have had experience in restaurant dining, walking into a restaurant will activate behaviors associated with waiting for the hostess or looking at the menu. As we review the menu, decision-making processes come into play: We may place a high “payoff” on a preferred dish such as baked stuffed lobster, but also consider the fresh sole when we see it is half the price and also won’t require wearing a lobster bib in front of our new date.

External inputs can be all it takes to trigger schema control units. For example, it is hard not to move your eyes when tracking a moving object. But in most situations, our actions are not dictated by the input alone; many schema control units can be activated at the same time, and so we need a control process to ensure that the appropriate control units are selected. Norman and Shallice postulated two types of selection processes. One is contention scheduling, which manages schemas for automatic or familiar actions. This process is fast, but passive. Although schemas are driven by perceptual inputs, they also compete with one another, especially when two control units are mutually exclusive. Contention scheduling, through its inhibitory connections between schemas, prevents competing actions and ensures that we act coherently. This is why we cannot look at two places at the same time, or move the same hand to pick up a glass and a fork simultaneously. Only one schema (or nonoverlapping schemas) can win the competition. If competition does not resolve the conflict, none of the schemas are activated enough to trigger a response, and the result is no action.

The second means of selection in this model comes by way of the supervisory attentional system (SAS), which can supersede contention scheduling. The SAS is essential for ensuring that behavior is flexible by allowing us to override automatic behavior. It is a mechanism for favoring certain schema control units to reflect the demands of the situation or to emphasize some goals over others.

The SAS is a psychological model of cognitive control. It specifies some of the key situations when control operations would be useful. We introduced these situations near the beginning of this chapter. Selection might benefit from the SAS in situations when

Although the SAS in this model (Figure 12.31) is sketched as a single entity, research over the past 30 years suggests that multiple neural structures are involved in all of these operations. The functions embodied in the SAS are part of a distributed network, a set of neural regions that, as a group, come into play in the situations we have described here.

The last four situations just listed share one aspect of cognitive control that has not been discussed in detail to this point. For a person engaged in goal-oriented behavior, especially a behavior that includes subgoals, it is important to have a way to monitor moment-to-moment progress. If this is a well-learned process, there should be a means for signaling deviations from the expected course of events.

One might expect the task of a monitoring system to be like that of a supervisor, keeping an eye on the overall flow of activity and being ready to step in whenever a problem arises. The head chef must attend to the actions of her staff to ensure the team’s activities are coordinated to produce the perfect meal. If the salad course is delayed because the prep cook has taken an extended break, the entire production can collapse. By monitoring the various components of the operation, the chef can make online adjustments, texting the prep chef to get back from his break or alerting the sous-chef to step in as the problem develops. For a neural monitoring system, however, there is a problem with this analogy: It has the feel of a homunculus. The head chef, or supervisor, has to have knowledge of the entire process and understand how the parts work together. A goal for any physiological model of cognitive control is, in one sense, the opposite: How can the kinds of simple operations that characterize neurons lead to cognitive control operations such as monitoring?

The last 30 years have witnessed burgeoning interest in the medial frontal cortex (MFC), especially the anterior cingulate cortex, as a critical component of a monitoring system. Moreover, the evolution of theoretical accounts of the MFC provides an especially interesting story within the history of cognitive neuroscience. Buried in the depths of the frontal lobes and characterized by a primitive cytoarchitecture, the cingulate cortex was assumed to be a component of the limbic system, helping to modulate autonomic responses during painful or threatening situations. The functional roles ascribed to most cortical regions have been inspired by behavioral problems associated with neurological disorders. Interest in the anterior cingulate, however, was sparked when serendipitous activations were seen in this region during some of the first neuroimaging studies.

Subsequent studies have revealed that the medial frontal cortex is consistently engaged whenever a task becomes more difficult, the type of situation in which monitoring demands are likely to be high. One meta-analysis highlighted the center of activation in 38 fMRI studies that included conditions in which monitoring demands were high. The activations were clustered in the anterior cingulate regions (areas 24 and 32) but also extended into areas 8 and 6; thus, we refer to this entire region as medial frontal cortex.

How Does Medial Frontal Cortex Monitor Processing in Cognitive Control Networks?

As with much of the frontal cortex, the medial frontal cortex exhibits extensive connectivity with much of the brain. For example, DTI studies suggest that there are at least 11 subregions just within the ACC (Figure 12.32). These subregions are defined by their distinct patterns of white matter connectivity with other brain regions. One region shows strong connectivity with OFC, another with ventral striatum, another with premotor cortex, and so on. This anatomy is consistent with the hypothesis that the medial frontal cortex is in a key position to influence decision making, goal-oriented behavior, and motor control. Making sense of the functional role of this region has proven to be an area of ongoing and lively debate. We now turn to some hypotheses that have been proposed to account for the functional role of the medial frontal cortex.

Attentional Hierarchy Hypothesis An early hypothesis centered on the idea that the medial frontal cortex should be conceptualized as part of an attentional hierarchy. In this view, the medial frontal cortex occupies an upper rung on the hierarchy, playing a critical role in coordinating activity across attentional systems (Figure 12.33). Consider a PET study of visual attention in which participants must selectively attend to a single visual dimension (color, motion, shape) or monitor changes in all three dimensions simultaneously. In the latter condition, attentional resources must be divided (Corbetta et al., 1991). Compared to control conditions in which stimuli were viewed passively, the selective-attention conditions were associated with enhanced activity in feature-specific regions of visual association areas. For example, attending to motion was correlated with greater blood flow in the lateral prestriate cortex, whereas attending to color stimulated blood flow in more medial regions. During the divided-attention task, however, the most prominent activation was in the anterior cingulate cortex. These findings suggest that selective attention causes local changes in regions specialized to process certain features. The divided-attention condition, in contrast, requires a higher-level attentional system—one that simultaneously monitors information across these specialized modules.

An association between the medial frontal cortex and attention is further shown by how activation in this region changes as attentional demands decrease. If the verb generation task (see Figure 12.23) is repeated over successive blocks, the primary activation shifts from the cingulate and prefrontal regions to the insular cortex of the temporal lobe (Raichle et al., 1994). This shift indicates that the task has changed. In the initial trial, participants have to choose between alternative semantic associates. If the target noun is apple, then possible responses are “peel,” “eat,” “throw,” or “juggle,” and the participant must select between these alternatives. On subsequent trials, however, the task demands change from semantic generation to memory retrieval. The same semantic associate is almost always reported. Thus, if a participant reports “peel” on the first trial, invariably he will make the same choice on subsequent trials.

Activation of the anterior cingulate during the first trial can be related to two of the functions of a supervisory attentional system (SAS): responding (a) under novel conditions and (b) with more difficult tasks. The generation condition is more difficult than the repeat condition because the response is not constrained. But over subsequent trials, the generation condition becomes easier (as evidenced by markedly reduced response times), and the items are no longer novel. Meanwhile, the elevated activation of the cingulate dissipates, reflecting a reduced need for the SAS. That this shift indicates the loss of novelty rather than a general decrease in the medial frontal cortex activity with practice is shown by the finding that, when a new list of nouns is used, the cingulate activation returns.

One concern with the hierarchy model is that it is descriptive rather than mechanistic. The model recognizes that the medial frontal cortex is recruited when attentional demands are high, but it does not specify how this recruitment occurs, nor does it specify the kinds of representations supported by the medial frontal cortex. We might suppose that the representation includes the current goal as well as all the suboperations required to achieve that goal. This type of representation, however, is quite complex. What’s more, even if all of this information were represented in the medial frontal cortex, we would still not be able to explain how it uses this information to implement cognitive control. In a sense, the hierarchical attention model is reminiscent of the homunculus problem: To explain control, we postulate a controller without describing how the controller is controlled.

Error Detection Hypothesis Concern about the attentional hierarchy hypothesis has led researchers to consider other models of how medial frontal cortex might be involved in monitoring behavior. The starting point for one model comes from evidence implicating medial frontal cortex in the detection of errors. Evokedpotential studies have shown that the medial frontal cortex provides an electrophysiological signal correlated with the occurrence of errors. When people make an incorrect response, a large evoked response sweeps over the prefrontal cortex just after the movement is initiated (Figure 12.34). This signal, referred to as the error-related negativity (ERN) response, has been localized to the anterior cingulate (Dehaene et al., 1994). It might be supposed that a monitoring system would detect when an error has occurred and that this information would be used to increase cognitive control.

This hypothesis provides a different perspective on the co-occurrence of activation in medial and lateral prefrontal cortex—one that captures many of the functional benefits of an attentional system. Typically, we make errors when we are not paying much attention to the task at hand. Consider being asked to perform the task shown in Figure 12.34 for an hour, during which the stimulus appears only once every 6 seconds. Pretty boring, right? At some point, your mind will start to wander. You might think about your evening plans. This new goal begins to occupy working memory, displacing the experimentally defined (boring) goal to respond to the letter in the center and not the letters on the side. Oops—you suddenly find yourself pressing the wrong key. Physiological responses such as the ERN could be used to reactivate the experimental goal in working memory.

One group of researchers (Eichele et al., 2008) used fMRI to see if they could predict when people were likely to make an error. They looked at the event-related response over successive trials, asking how the signals changed in advance of an error. Two changes were especially notable. First, before an error was made, the researchers observed a steady decrease in activity within a network spanning medial frontal cortex and right inferior frontal cortex, a decrease that could be detected up to 30 s before the error (Figure 12.35). Second, activity increased over a similar time period in the precuneus and retrosplenial cortex. These two regions are key components of the default network, which is postulated to be associated with self-referential processing (e.g., when you start to think about something other than the task at hand; see Chapter 13). Thus, we can see a shift in activity from the monitoring system to the mind-wandering system, which builds until a person makes an error.

The ERN is an especially salient signal of a monitoring system. The engagement of medial frontal cortex, however, is not limited to conditions in which people make errors. Medial frontal cortex activation is also prominent in many tasks in which errors rarely occur. The Stroop task is one such example. The difficulty that people have when the words and colors are incongruent is typically detected in the reaction time data and only minimally, if at all, in measures of accuracy. That is, people take longer, but they don’t make mistakes. Still, activation of the medial frontal cortex is much higher on incongruent trials compared to when the words and colors are congruent (Bush et al., 2000). Similarly, activation is higher when people are asked to generate the verbs associated with nouns compared to when they just repeat the nouns, even though errors rarely occur.

Response Conflict Hypothesis Jonathan Cohen and his colleagues (2000) at Princeton University have hypothesized that a key function of the medial frontal cortex is to evaluate response conflict. This hypothesis is intended to provide an umbrella account of the monitoring role of this region, encompassing earlier models that focused on attentional hierarchies or error detection. Difficult and novel situations should engender high response conflict. In the verb generation task, there is a conflict between acceptable alternative responses. Errors, by definition, are also situations in which conflict exists. Similarly, tasks such as the Stroop task entail conflict in that the required response is in conflict with a more habitual response. In Cohen’s view, conflict monitoring is a computationally appealing way to allocate attentional resources. When the monitoring system detects that conflict is high, there is a need to increase attentional vigilance. Increases in anterior cingulate activity can then be used to modulate activity in other cortical areas.

Event-related fMRI has been used to pit the error detection hypothesis against the conflict-monitoring hypothesis. One study used the flanker task, similar to that shown in Figure 12.34, except that the letters were replaced by a row of five arrows (Botvinick et al., 1999). Participants responded to the direction of the central arrow, pressing a button on the right side if this arrow pointed to the right and pressing a button on the left side if this arrow pointed to the left. On compatible trials, the flanking arrows pointed in the same direction; on incompatible trials, the flanking arrows pointed in opposite directions. Neural activity in the medial frontal cortex was higher on the incompatible trials compared to the compatible trials. Importantly, this increase was observed even when participants responded correctly. These results strongly suggest that the monitoring demands, and not the occurrence of an error, engage the medial frontal cortex.

Subsequent work has sought to clarify how a conflictmonitoring process might be part of a network for cognitive control. Consider a variant of the Stroop task in which a cue is presented at the beginning of each trial to indicate whether the participant should read the word or name the color. After a delay, the cue is replaced by a Stroop stimulus. By using a long delay between the cue and stimulus, researchers can separately examine the neural responses related to goal selection and the neural responses related to response conflict. Moreover, by using a cue, the design allows the experimenters to manipulate two factors: (a) goal difficulty, given the assumption that it is easier to read words than to name their ink color; and (b) color–word congruency.

The results showed distinct neural correlates of these two factors (Figure 12.36). The degree of difficulty for goal selection was evident in the activation of the lateral prefrontal cortex. When the task was made more difficult, the BOLD response in this region increased even before the actual stimulus was presented. In contrast, activation in the medial frontal cortex was sensitive to the degree of response conflict, being greater when the word and stimulus color were different. The picture here is similar to that observed in the ERN literature. The lateral prefrontal cortex represents the task goal, and the medial frontal cortex monitors whether that goal is being achieved. One difference from the error detection model, though, is that the medial monitoring process is recruited not just when errors occur. Rather, it is engaged whenever there is conflict—which we would expect to be quite high in novel contexts or with particularly demanding tasks.

Note that the preceding study shows only the distinct contributions of the lateral and medial frontal regions. A subsequent event-related fMRI study provided direct evidence that these two regions work in tandem to provide cognitive control. Activation in the lateral prefrontal cortex was highly correlated with activation in the medial frontal cortex on the preceding trial (Figure 12.36c). Thus a signal of high response conflict on an incongruent Stroop trial led to a stronger response in the lateral prefrontal cortex. As with the error model, the medial monitoring function can be used to modulate the activation of the goal in working memory. Difficult trials help remind the person to stay on task. We can hypothesize that the medial frontal activity modulates filtering operations of the prefrontal cortex, ensuring that the irrelevant word names are ignored. Interestingly, activation in the medial frontal cortex on incongruent trials was lower when the previous trial was also incongruent. Assuming that an incongruent trial leads to a stronger activation of the task goal in working memory and, as a result, there is better filtering of irrelevant information on the next trial, the degree of conflict generated on that trial will decrease.

ACC Function Is Still Up in the Air The conflict-monitoring hypothesis remains a work in process, and the literature suggests some problems that need to be addressed. For example, activation in the anterior cingulate is more closely linked with the participant’s anticipation of possible errors than with the degree of conflict (J. W. Brown & Braver, 2005). This result suggests that the medial frontal cortex may be doing more than simply monitoring the level of conflict presented by the current environment. It may also be anticipating the likelihood of conflict, suggesting a risk prediction and error avoidance role. As seen in our earlier discussion of decision making, the cingulate cortex has been linked to evaluating the effort associated with a behavioral choice, helping to perform a cost–benefit analysis. This hypothesis has led to a reinterpretation of the prevalent activation of medial frontal cortex observed on difficult tasks. Jack Grinband and his colleagues (2008) observed that the response times tend to be larger in such conditions. They suggested that the activation here may simply reflect the amount of time spent on the task, a variant of effort. To test this idea, they had participants view a checkerboard that flashed on and off for a variable duration of time, and simply press a button when the checkerboard disappeared. In this task, the stimulus is unambiguous, only one response is possible, and no choice decision is required. Thus, there were no errors, nor is there any conflict. Even so, medial frontal cortex activation was modulated by task duration and was similar to that observed when the participants performed a Stroop task. It is, of course, hard to make inferences about a null result (similar activation in these two tasks), but the results provide an alternative view on why activation of medial frontal cortex is correlated with task difficulty.

More perplexing are the results of studies involving patients with lesions of the medial frontal cortex. These patients show little evidence of impairment on various tasks that would appear to require cognitive control, one reason why the cingulate had not been identified as having a role until fMRI came along. For example, these patients are as sensitive to the effects of an error on the Stroop task as are control participants (Fellows & Farah, 2005). In fact, the patient data fail to confirm a number of behavioral predictions derived from models of how medial frontal function contributes to cognitive control. Although their cognitive performance appears to be relatively normal, these patients do exhibit a marked impairment: They fail to show normal changes in arousal when challenged either physically through exercise or mentally with math problems (Critchley et al., 2003). This finding suggests that medial frontal cortex may play a regulatory role in modulating autonomic activity in response to the current context, providing an interface between cognition and arousal. This modulation would be an indirect form of control, linked to regulatory mechanisms in the brainstem rather than through direct interactions with the cognitive representations of prefrontal cortex.

Importantly, the error detection and conflict-monitoring hypotheses suggest a way to achieve rather sophisticated control without resorting to homunculus-like notions. It is possible to envision a rather simple neural mechanism that assesses the degree to which multiple responses are concurrently active. Whether these ideas prove to have lasting value, they do offer an encouraging example of how even the most advanced of our cognitive competencies can be subject to rigorous experimental investigation, given the many tools of cognitive neuroscience.

Summary

The prefrontal cortex plays a crucial role in cognitive control functions that are critical for goal-oriented behavior and decision making. Cognitive control systems allow us to be flexible and not driven solely by automatic behavior. The prefrontal cortex contains a massively connected network linking the brain’s motor, perceptual, and limbic regions and is in an excellent position to coordinate processing across wide regions of the central nervous system (CNS).

Goal-oriented behavior and decision making involve planning, evaluating options, and calculating the value of rewards and consequences. These behaviors require that we represent information that is not always immediately present in the environment. Working memory is essential for this function. It allows for the interaction of current goals with perceptual information and knowledge accumulated from personal experience. Not only must we be able to represent our goals, but these representations must persist for an extended period of time. Working memory must be dynamic. It requires the retrieval, amplification, and manipulation of representations that are useful for the task at hand as well as the ability to ignore potential distractions. Yet we must also be flexible. If our goals change, or if the context demands an alternative course of action, we must be able to switch from one plan to another. These operations require a system that can monitor ongoing behavior, signaling when we fail or when there are potential sources of conflict.

Two functional systems have been emphasized in this chapter: (a) The lateral prefrontal cortex and frontal pole support goal-oriented behavior, providing a working memory system that recruits and selects task-relevant information stored in the more posterior regions of the cortex. (b) The medial frontal cortex is hypothesized to work in tandem with the prefrontal cortex, monitoring ongoing activity so as to be able to modulate the degree of cognitive control. As we emphasized in this chapter and in Chapter 8, the control of action has a hierarchical nature. Just as control in the motor system is delegated across many functional systems, an analogous organization characterizes prefrontal function. With control distributed in this manner, the need for an all-powerful controller, a homunculus, is minimized.

The content of ongoing processing is embedded in a context that reflects the history and current goals of the actor. Up to now, we have focused on relatively impersonal goals: naming words, attending to colors, remembering locations. But most of our actions are socially oriented. They reflect our personal desires, both as individuals and as members of social groups. To gain a more complete appreciation of goal-oriented behavior, we must turn to the study of the social brain, asking how our behavior is influenced by our interactions with others. In Chapter 13 we will address this topic, with the spotlight focusing on the ventromedial prefrontal cortex. By recognizing the intimate connections between the regions of prefrontal cortex, we can start to appreciate how a mind emerges from the architecture of the human brain.

Key Terms

action–outcome (p. 511)

action–outcome decision (p. 521)

anterior cingulate cortex (p. 550)

cognitive control (p. 508)

delayed-response task (p. 512)

descriptive decision theory (p. 521)

dopamine (p. 510)

dynamic filtering (p. 535)

error-related negativity (ERN) (p. 552)

frontal pole (FP) (p. 509)

goal-oriented action (p. 511)

goal-oriented behavior (p. 508)

habit (p. 511)

inhibitory control (p. 541)

lateral prefrontal cortex (LPFC) (p. 509)

medial frontal cortex (MFC) (p. 509)

mesocortical pathway (p. 526)

monitoring (p. 550)

normative decision theory (p. 520)

nucleus accumbens (p. 526)

orbitofrontal cortex (OFC) (p. 509)

perseveration (p. 510)

prediction error (PE) (p. 527)

prefrontal cortex (PFC) (p. 509)

primary reinforcer (p. 522)

recency memory (p. 512)

response conflict (p. 553)

secondary reinforcer (p. 522)

stimulus–response decision (p. 521)

striatum (p. 510)

supervisory attentional system (SAS) (p. 550)

utilization behavior (p. 511)

value (p. 522)

ventral tegmental area (p. 510)

working memory (p. 512)

Thought Questions

Describe three examples from your daily activities that demonstrate how actions involve the interplay of habit-like behaviors and goal-oriented behaviors.
What are some of the current hypotheses concerning functional specialization across the three gradients on the frontal cortex (anterior-posterior, dorsal-ventral, lateral-medial)?
A cardinal feature of human cognition is that we exhibit great flexibility in our behavior. Flexibility implies choice, and choice entails decision making. Describe some of the neural systems involved in decision making.
Review and contrast some of the ways in which the prefrontal cortex and the medial frontal cortex are involved in monitoring and controlling processing.
The notion of a supervisory attentional system does not sit well with some researchers, because it seems like a homuncular concept. Is such a system a necessary part of a network for cognitive control? Explain your answer.