Chapter 07: Attention

A PATIENT, WHO HAD a severe stroke several weeks earlier, sits with his wife as she talks with his neurologist. Although at first it seemed that the stroke had left him totally blind, his wife states that he can sometimes see things. They are hoping his vision will improve. The neurologist soon realizes that her patient’s wife is correct. The man does have serious visual problems, but he is not completely blind. Taking a comb from her pocket, the doctor holds it in front of her patient and asks him, “What do you see?” (Figure 7.1a).

“Well, I’m not sure,” he replies, “but... oh... it’s a comb, a pocket comb.”

“Good,” says the doctor. Next she holds up a spoon and asks the same question (Figure 7.1b).

The doctor nods and then holds up the spoon and the comb together. “What do you see now?” she asks.

“Okay...,” she says as she overlaps the spoon and comb in a crossed fashion so they are both visible in the same location. “What do you see now?” (Figure 7.1c). Oddly enough, he sees only the comb. “What about a spoon?” she asks.

“Nope, no spoon,” he says, but then suddenly blurts out, “Yes, there it is, I see the spoon now.”

Shaking the spoon and the comb vigorously in front of her patient’s face, the doctor persists, “You don’t see anything else, nothing at all?”

He stares straight ahead, looking intently, and finally says, “Yes... yes, I see them now... I see some numbers.”

“Yes,” he squints and appears to strain his vision, moving his head ever so slightly, and replies, “I see numbers.” The doctor then notices that the man’s gaze is directed to a point beyond her and not toward the objects she is holding. Turning to glance over her own shoulder, she spots a large clock on the wall behind her!

FIGURE 7.1 Examination of a patient recovering from a cortical stroke.
(a) The doctor holds up a pocket comb and asks the patient what he sees. The patient reports seeing the comb. (b) The doctor then holds up a spoon, and the patient reports seeing the spoon too. (c) But when the doctor holds up both the spoon and the comb at the same time, the patient says he can see only one object at a time. The patient has Bálint’s syndrome.

Even though the doctor is holding both objects in one hand directly in front of her patient, overlapping them in space and in good lighting, he sees only one item at a time. That one item may even be a different item altogether: one that is merely in the direction of his gaze, such as the clock on the wall. The neurologist diagnoses the patient: He has Bálint’s syndrome, first described in the late 19th century by the Hungarian neurologist and psychiatrist Rezső Bálint. It is a severe disturbance of visual attention and awareness, caused by bilateral damage to regions of the posterior parietal and occipital cortex. The result of this attention disturbance is that only one or a small subset of available objects are perceived at any one time and are mislocalized in space. The patient can “see” each of the objects presented by the doctor—the comb, the spoon, and even the numbers on the clock. He fails, however, to see them all together and cannot accurately describe their locations with respect to each other or to himself.

Bálint’s syndrome is an extreme pathological instance of what we all experience daily: We are consciously aware of only a small bit of the vast amount of information available to our sensory systems from moment to moment. By looking closely at patients with Bálint’s syndrome and the lesions that cause it, we have come to learn more about how, and upon what, our brain focuses attention. The central problem in the study of attention is how the brain is able to select some information at the expense of other information.

Robert Louis Stevenson wrote, “The world is full of a number of things, I’m sure we should all be as happy as kings.” Although those things may make us happy, the sheer number of them presents a problem to our perception system: information overload. We know from experience that we are surrounded by more information than we can handle and comprehend at any given time. The nervous system, therefore, has to make “decisions” about what to process. Our survival may depend on which stimuli are selected and in what order they are prioritized for processing. Selective attention is the ability to prioritize and attend to some things while ignoring others. What determines the priority? Many things. For instance, an optimal strategy in many situations is to attend to stimuli that are relevant to current behavior and goals. For example, to survive this class, you need to attend to this chapter rather than your Facebook page. This is goal-driven control (also called top-down control) driven by an individual’s current behavioral goals and shaped by learned priorities based on personal experience and evolutionary adaptations. Still, if you hear a loud bang, even while dutifully attending this book, you reflexively pop up your head and check it out. That is good survival behavior because a loud noise may presage danger. Your reaction was stimulus driven and is therefore termed stimulus-driven control (also known as bottom-up or reflexive control), which is much less dependent on current behavioral goals.

Attention grabbed the attention of William James (Figure 7.2). At the end of the 19th century, this great American psychologist made an astute observation:

In this insightful quote, James has captured key characteristics of attentional phenomena that are under investigation today. For example, his statement “it is the taking possession by the mind” suggests that we can choose the focus of attention; that is, it can be voluntary. His mention of “one out of what seem several simultaneously possible objects or trains of thought” refers to the inability to attend to many things at once, and hence the selective aspects of attention. James raises the idea of limited capacity in attention, by noting that “it implies withdrawal from some things in order to deal effectively with others.”

As clear and articulate as James’s writings were, little was known about the behavioral, computational, or neural mechanisms of attention during his lifetime. Since then, knowledge about attention has blossomed, and researchers have identified multiple types and levels of attentive behavior. First, let’s distinguish selective attention from arousal. Arousal refers to the global physiological and psychological state of the organism. Our level of arousal is the point where we fall on the continuum from being hyperaroused (such as during periods of intense fear) to moderately aroused (which must describe your current state as you start to read about the intriguing subject of attention) to groggy (when you first got up this morning) to lightly sleeping to deeply asleep.

Selective attention, on the other hand, is not a global brain state. Instead, it is how—at any level of arousal—attention is allocated among relevant inputs, thoughts, and actions while simultaneously ignoring irrelevant or distracting ones. As shorthand, we will use the term attention when referring to the more specific concept of selective attention. Attention influences how people code sensory inputs, store that information in memory, process it semantically, and act on it to survive in a challenging world. This chapter focuses on the mechanisms of selective attention and its role in perception and awareness.

Mechanisms that determine where and on what our attention is focused are referred to as attentional control mechanisms. They involve widespread, but highly specific, brain networks. These attentional control mechanisms influence specific stages of information processing, where it is said that “selection” of inputs (or outputs) takes place—hence the term selective attention. In this chapter, we first review the anatomical structures involved in attention. Then, we consider how damage to the brain changes human attention and gives us insights into how attention is organized in the brain. Next, we discuss how attention influences sensation and perception. We conclude with a discussion of the brain networks used for attentional control.

Our attention system uses subcortical and cortical networks within the brain that interact to enable us to selectively process information in the brain.

Several subcortical structures are relevant to both attentional control and selection. The superior colliculus in the midbrain and the pulvinar are involved in aspects of the control of attention. We know that damage to these structures can lead to deficits in the ability to orient overt (i.e., eye gaze) and covert (i.e., attention directed without changes in eyes, head, or body orientation) attention. Within the cortex are several areas that are important in attention—portions of the frontal cortex, posterior parietal cortex, and posterior superior temporal cortex as well as more medial brain structures including the anterior cingulate cortex, the posterior cingulate cortex, and insula. Cortical and subcortical areas involved in controlling attention are shown in the Anatomical Orientation box. As we will learn, cortical sensory regions are also involved, because attention affects how sensory information is processed in the brain.

Much of what neuroscientists know about brain attention systems has been gathered by examining patients who have brain damage that influences attentional behavior. Many disorders result in deficits in attention, but only a few provide clues to which brain systems are being affected. Some of the best-known disorders of attention (e.g., attention deficit/hyperactivity disorder, or ADHD) are the result of disturbances in neural processing within brain attention systems. The portions of the brain’s attention networks affected by ADHD have only recently begun to be identified.

In contrast, important information has been derived about attentional mechanisms and the underlying neuroanatomical systems supporting attention, by investigating classic syndromes like “unilateral spatial neglect” (described next) and Bálint’s syndrome. These disorders are the result of focal brain damage (e.g., stroke) that can be mapped in postmortem analyses and with brain imaging in the living human. Let’s consider how brain damage has helped us understand brain attention mechanisms.

Unilateral spatial neglect, or simply neglect, results when the brain’s attention network is damaged in only one hemisphere. The damage typically occurs from a stroke and, unfortunately, is quite common. Although either hemisphere could be affected, the more severe and persistent effects occur when the right hemisphere is damaged. Depending on the severity of the damage, its location, and how much time has passed since the damage occurred, patients may have reduced arousal and processing speeds, as well as an attention bias in the direction of their lesion (ipsilesional). For example, a right-hemisphere lesion would bias attention toward the right, resulting in a neglect of what is going on in the left visual field. Careful testing can show that these symptoms are not the result of partial blindness, as we will describe later. A patient’s awareness of his lesion and deficit can be severely limited or lacking altogether. For instance, patients with right-hemisphere lesions may behave as though the left regions of space and the left parts of objects simply do not exist. If you were to visit a neglect patient and enter the room from the left, he might not notice you. He may have groomed only the right side of his body, leaving half his face unshaved and half his hair uncombed. If you were to serve him dinner, he may eat only what is on the right side of his plate; when handed a book, he may read only the right-hand page. What’s more, he may deny having any problems. Such patients are said to “neglect” the information.

A graphic example of neglect is seen in paintings by the late German artist Anton Raederscheidt. At age 67, Raederscheidt suffered a stroke in the right hemisphere, which left him with neglect. The pictures in Figure 7.3 are self-portraits that he painted at different times after the stroke occurred and during his partial recovery. The paintings show his failure to represent portions of contralateral space—including, remarkably, portions of his own face. Notice in the first painting (Figure 7.3a), done shortly after his stroke, that almost the entire left half of the canvas is untouched. The image he paints of himself, in addition to being poorly formed, is missing the left half. The subject has one eye, part of a nose, and one ear; toward the left, the painting fades away. In each of the next three paintings (Figure 7.3b–d), made over the following several weeks and months, Raederscheidt uses more and more of the canvas and includes more and more of his face, until in Figure 7.3d, he uses most of the canvas. He now has a bilaterally symmetrical face, although some minor asymmetries persist in his painting.

Typically, patients show only a subset of these extreme signs of neglect, and indeed, neglect can manifest itself in different ways. The common thread is that, despite normal vision, neglect involves deficits in attending to and acting in the direction that is opposite the side of the unilateral brain damage. One way to observe this phenomenon is to look at the patterns of eye movements in patients with neglect. Figure 7.4 (top) shows eye movement patterns in a patient with a right hemisphere lesion and neglect during rest and when searching a bilateral visual array for a target letter. The patient’s eye movements are compared to those of patients with right hemisphere strokes who showed no signs of neglect (Figure 7.4, bottom). The neglect patient shows a pattern of eye movements that are biased in the direction of the right visual field, while those without neglect search the entire array, moving their eyes equally to the left and right.

To diagnose neglect of contralesional space, neuropsychological tests are used. In the line cancellation test, patients are given a sheet of paper containing many horizontal lines and are asked to bisect the lines precisely in the middle by drawing a vertical line. Patients with lesions of the right hemisphere and neglect tend to bisect the lines to the right of the midline. They may also completely miss lines on the left side of the paper (Figure 7.5). In this example, the pattern of line cancellation is evidence of neglect at the level of object representations (each line) as well as visual space (the visual scene represented by the test paper).

A related test is copying objects or scenes. When asked to copy a simple line drawing, such as a flower or clock face, patients with neglect have difficulty. Figure 7.6 shows an example from a patient with a right-hemisphere stroke who was asked to copy a clock. Like the artist Raederscheidt, the patient shows an inability to draw the entire object and tends to neglect the left side. Even when they know and can state that clocks are round and include numbers 1 to 12, they cannot properly copy the image or draw it from memory.

So far, we have considered neglect for items that are actually present in the visual world. But neglect can also affect the imagination and memory. Eduardo Bisiach and Claudio Luzzatti (1978) studied patients with neglect caused by unilateral damage to their right hemisphere. They asked their patients, who were from Milan, to imagine themselves standing on the steps of the Milan Cathedral (the Duomo di Milano) and to describe from memory the piazza (church square) from that viewpoint. Amazingly, the patients neglected things on the side of the piazza contralateral to their lesion, just as if they were actually standing there looking at it. When the researchers next asked the patients to imagine themselves standing across the piazza, facing toward the Duomo, they reported items from visual memory that they had previously neglected, and neglected the side of the piazza that they had just described (Figure 7.7).

Thus, neglect is found for items in visual memory during remembrance of a scene as well as for items in the external sensory world. The key point in the Bisiach and Luzzatti experiment is that the patients’ neglect could not be attributed to lacking memories but rather indicated that attention to parts of the recalled images was biased.

How do we distinguish neglect from blindness in the contralateral visual hemifields? Well, visual field testing can show that neglect patients detect stimuli normally when those stimuli are salient and presented in isolation. For example, when simple flashes of light or the wiggling fingers of a neurologist are shown at different single locations within the visual field of a neglect patient, he can see all the stimuli, even those that are in the contralateral (neglected) hemifield. This result tells us that the patient does not have a primary visual deficit. The patient’s neglect becomes obvious when he is presented simultaneously with two stimuli, one in each hemifield. In that case, the patient fails to perceive or act on the contralesional stimulus. This result is known as extinction, because the presence of the competing stimulus in the ipsilateral hemifield prevents the patient from detecting the contralesional stimulus. With careful testing, doctors often can see residual signs of extinction, even after the most obvious signs of neglect have remitted as a patient recovers. Figure 7.8 shows a neurologist testing a patient with right parietal damage in order to investigate his vision, and to reveal his neglect by showing extinction.

It’s important to realize that these biases against the contralesional sides of space and objects can be overcome if the patient’s attention is directed to the neglected locations of items. This is one reason the condition is described as a bias, rather than a loss of the ability to focus attention contralesionally.

One patient’s comments help us understand how these deficits might feel subjectively: “It doesn’t seem right to me that the word neglect should be used to describe it. I think concentrating is a better word than neglect. It’s definitely concentration. If I am walking anywhere and there’s something in my way, if I’m concentrating on what I’m doing, I will see it and avoid it. The slightest distraction and I won’t see it” (Halligan & Marshall, 1998).

Let’s compare the pattern of deficits in neglect with those of the patient with Bálint’s syndrome, described at the beginning of this chapter. In contrast to the patient with neglect, a Bálint’s patient demonstrates three main deficits that are characteristic of the disorder: simultanagnosia, ocular apraxia, and optic ataxia.

Simultanagnosia is difficulty perceiving the visual field as a whole scene, such as when the patient saw only the comb or the spoon, but not both at the same time. Ocular apraxia is a deficit in making eye movements (saccades) to scan the visual field, resulting in the inability to guide eye movements voluntarily. When the physician overlapped the spoon and comb in space (see Figure 7.1), the Bálint’s patient should have been able, given his direction of gaze, to see both objects, but he could not. Lastly, Bálint’s patients also suffer from optic ataxia, a problem in making visually guided hand movements. If the doctor had asked the Bálint’s patient to reach out and grasp the comb, he would have had a difficult time reaching through space to grasp the object.

Both neglect and Bálint’s syndrome include severe disturbances in perception. The patterns of perceptual deficits are quite different, however, because different brain areas are damaged in each disorder. Neglect is the result of unilateral lesions of the parietal, posterior temporal, and frontal cortex. Neglect also can be due to damage in subcortical areas including the basal ganglia, thalamus, and midbrain. Bálint’s patients suffer from bilateral occipitoparietal lesions. Thus, researchers obtain clues about the organization of the brain’s attention system by considering the location of the lesions that cause these disorders and the differing perceptual and behavioral results. Neglect shows us that a network of cortical and subcortical areas, especially in the right hemisphere, result in disturbances of spatial attention. Bálint’s syndrome shows us that posterior parietal and occipital damage to both hemispheres leads to an inability to perceive multiple objects in space, which is necessary to create a scene.

What else can we understand about attention by contrasting patients with neglect to those with Bálint’s syndrome? From patients with neglect, we understand that the symptoms involve biases in attention based on spatial coordinates, and that these coordinates can be described in different reference frames. Put another way, neglect can be based on spatial coordinates either with respect to the patient (egocentric reference frame) or with respect to an object in space (allocentric reference frame). This finding tells us that attention can be directed within space and also within objects. Most likely these two types of neglect are guided by different processes. Indeed, the brain mechanisms involved with attending objects can be affected even when no spatial biases are seen. This phenomenon is seen in patients with Bálint’s syndrome, who have relatively normal visual fields but cannot attend to more than one or a few objects at a time, even when the objects overlap in space.

The phenomenon of extinction in neglect patients suggests that sensory inputs are competitive, because when two stimuli presented simultaneously compete for attention, the one in the ipsilesional hemifield will win the competition and reach awareness. Extinction also demonstrates that after brain damage, patients experience reduced attentional capacity: When two competing stimuli are presented at once, the neglect patient is aware of only one of them.

It is important to note that none of these attentional deficits are the result of damage to the visual system per se, because the patient is not simply blind. These observations from brain damage and resultant attentional problems set the stage for us to consider several questions:

To answer these questions, let’s look next at the cognitive and neural mechanisms of attention.

When people turn their attention to something, the process is called orienting. The concept of orienting our selective attention can be divided into two categories: voluntary attention and reflexive attention. Voluntary attention is our ability to intentionally attend to something, such as this book. It is a goal-driven process, meaning that goals, knowledge, or expectations are used to guide information processing. Reflexive attention is a bottom-up, stimulus-driven process in which a sensory event—maybe a loud bang, the sting of a mosquito, a whiff of garlic, a flash of light or motion—captures our attention. As we will see later in this chapter, these two forms of attention differ in their properties and perhaps partly in their neural mechanisms.

Attentional orienting also can be either overt or covert. We all know what overt attention is—when you turn your head to orient toward a stimulus, whether it is for your eyes to get a better look, your ears to pick up a whisper, or your nose to sniff the frying bacon—you are exhibiting overt attention. You could appear to be reading this book, however, while actually paying attention to the two students whispering at the table behind you. This behavior is covert attention.

In 1894, Hermann von Helmholtz (Figure 7.9a) performed a fascinating experiment in visual perception. He constructed a screen on which letters were painted at various distances from the center (Figure 7.9b). He hung the screen at one end of his lab and then turned off all the lights to create a completely dark environment. Helmholtz then used an electric spark to make a flash of light that briefly illuminated the screen. His goal was to investigate aspects of visual processing when stimuli were briefly perceived. As often happens in science, however, he stumbled on an interesting phenomenon.

Helmholtz noted that the screen was too large to view in its entirety without moving his eyes. Nonetheless, even when he kept his eyes fixed right at the center of the screen, he could decide in advance where he would pay attention: He made use of his covert attention. As we noted in the introduction to this section, covert means that the location he directed his attention toward could be different from the location toward which he was looking. Through these covert shifts of attention, Helmholtz observed that during the brief period of illumination, he could perceive letters located within the focus of his attention better than letters that fell outside the focus of his attention, even when his eyes remained directed toward the center of the screen.

Try this yourself using Figure 7.9. Hold the textbook 12 inches in front of you and stare at the plus sign in the center of Helmholtz’s array of letters. Now, without moving your eyes from the plus sign, read out loud the letters closest to the plus sign in a clockwise order. You have covertly focused on the letters around the plus sign. As Helmholtz wrote in his Treatise on Physiological Optics (translated into English in 1924), “These experiments demonstrated, so it seems to me, that by a voluntary kind of intention, even without eye movements, and without changes of accommodation, one can concentrate attention on the sensation from a particular part of our peripheral nervous system and at the same time exclude attention from all other parts.”

In the mid 20th century, experimental psychologists began to develop methods for quantifying the influence of attention on perception and awareness. Models of how the brain’s attention system might work were built from these data and from observations like those of Helmholtz—and from everyday experiences, such as attending a Super Bowl party.

Imagine yourself at a Super Bowl party having a conversation with a friend. How can you focus on this single conversation while the TV is blasting and boisterous conversations are going on around you? British psychologist E. C. Cherry (1953) wondered the same thing while attending cocktail parties. His curiosity and subsequent research helped to found the modern era of attention studies, with what was dubbed the cocktail party effect.

Selective auditory attention allows you to participate in a conversation at a busy bar or party while ignoring the rest of the sounds around you. By selectively attending, you can perceive the signal of interest amid the other noises. If, however, the person you are conversing with is boring, then you can give covert attention to a conversation going on behind you while still seeming to focus on the conversation in front of you (Figure 7.10).

Cherry investigated this ability by designing a cocktail party in the lab: Normal participants, wearing headphones, listened to competing speech inputs to the two ears—this setup is referred to as dichotic listening. Cherry then asked the participants to attend to and verbally “shadow” the speech (immediately repeat each word) coming into one ear, while simultaneously ignoring the input to the other ear. Cherry discovered that under such conditions, participants could not (mostly) report any details of the speech in the unattended ear (Figure 7.11). In fact, all they could reliably report from the unattended ear was whether the speaker was male or female. Attention, in this case voluntary attention, affected what was processed. This finding led Cherry and others to propose that attention to one ear results in better encoding of the inputs to the attended ear and loss or degradation of the unattended inputs to the other ear. You experience this type of thing when the person sitting next to you in lecture whispers a juicy tidbit in your ear. A moment later, you realize that you just missed what the lecturer said, although you could just as well have heard him with your other ear. As foreshadowed by William James, information processing bottlenecks seem to occur at stages of perceptual analysis that have a limited capacity. What is processed are the high-priority inputs that you selected. Many processing stages take place between the time information enters the eardrum and you become aware of speech. At what stages do these bottlenecks exist such that attention is necessary to favor the attended over the unattended signals?

This question has been difficult to answer. It has led to one of the most debated issues in psychology over the past five decades. Are the effects of selective attention evident early in sensory processing or only later, after sensory and perceptual processing are complete? Think about this question differently: Does the brain faithfully process all incoming sensory inputs to create a representation of the external world, or can processes like attention influence sensory processing? Is what you perceive a combination of what is in the external world and what is going on inside your brain? By “going on inside your brain,” we mean what your current goals may be, and what knowledge is stored in your brain. Consider the example in Figure 7.12. The first time you look at this image, you won’t see the Dalmatian dog in the black-and-white scene; you cannot perceive it easily. Once it is pointed out to you, however, you perceive the dog whenever you are shown the picture. Something has changed in your brain, and it is not simply knowledge that it is a photo of dog—the dog jumps out at you, even when you forget having seen the photo before. This is an example of the knowledge stored in your brain influencing your perception. Perhaps it is not either-or; it may be that attention affects processing at many steps along the way from sensory transduction to awareness.

Cambridge University psychologist Donald Broadbent (1958) elaborated on the idea that the information processing system has a limited-capacity stage or stages through which only a certain amount of information can pass (Figure 7.13)—that is, a bottleneck, as hinted at by the writings of James and the experiments of Cherry. In Broadbent’s model, the sensory inputs that can enter higher levels of the brain for processing are screened so that only the “most important,” or attended, events pass through. Broadbent described this mechanism as a gate that could be opened for attended information and closed for ignored information. Broadbent argued for information selection early in the information processing stream. Early selection, then, is the idea that a stimulus can be selected for further processing, or it can be tossed out as irrelevant before perceptual analysis of the stimulus is complete.

In contrast, models of late selection hypothesize that all inputs are processed equally by the perceptual system. Selection follows to determine what will undergo additional processing, and perhaps what will be represented in awareness. The late-selection model implies that attentional processes cannot affect our perceptual analysis of stimuli. Instead, selection takes place at higher stages of information processing that involve internal decisions about whether the stimuli should gain complete access to awareness, be encoded in memory, or initiate a response. (The term decisions in this context refers to nonconscious processes, not conscious decisions made by the observer.) Figure 7.14 illustrates the differential stages of early versus late selection.

The original “all or none” early selection models, exemplified by gating models, quickly ran into a problem. Cherry observed in his cocktail party experiments that sometimes salient information from the unattended ear was consciously perceived, for example, when the listener’s own name or something very interesting was included in a nearby conversation. The idea of a simple gating mechanism, which assumed that ignored information was completely lost, could not explain this experimental finding. Anne Treisman (1969), now at Princeton University, proposed that unattended channel information was not completely blocked from higher analysis but was degraded or attenuated instead—a point Broadbent agreed with. Thus, early-selection versus lateselection models were modified to make room for the possibility that information in the unattended channel could reach higher stages of analysis, but with greatly reduced signal strength. To test these competing models of attention, researchers employed increasingly sensitive methods for quantifying the effects of attention. Their methods included chronometric analysis—the analysis of the time course of information processing on a millisecond-tomillisecond level of resolution, as described next.

One way of measuring the effect of attention on information processing is to examine how participants respond to target stimuli under differing conditions of attention. Various experimental designs have been used for these explorations, and we describe some of them later in this chapter. One popular method is to provide cues that direct the participant’s attention to a particular location or target feature before presenting the task-relevant target stimulus. In these so-called cuing tasks, the focus of attention is manipulated by the information in the cue.

In cuing studies of voluntary spatial attention, participants are presented a cue that directs their attention to one location on a video screen (Figure 7.15). Next, a target stimulus is flashed onto the screen at either the cued location or another location. Participants may be asked to press a button as fast as they can following the presentation of a target stimulus to indicate that it occurred; or they may be asked to respond to something about the stimulus, such as, “was it red or blue?” Such designs can provide information on how long it takes to perform the task (reaction time or response time), how accurately the participant performs the task, or both. In one version of this experiment, participants are instructed that although the cue, such as an arrow, will indicate the most likely location of the upcoming stimulus, they are to respond to the target wherever it appears. The cue, therefore, predicts the location of the target on most trials (a trial is one presentation of the cue and subsequent target, along with the required response). This form of cuing is known as endogenous cuing. Here, the orienting of attention to the cue is driven by the participant’s voluntary compliance with the instructions and the meaning of the cue, rather than merely by the cue’s physical features (see Reflexive Attention, later in this chapter, for a contrasting mechanism).

When a cue correctly predicts the location of the subsequent target, we say we have a valid trial (Figure 7.15a). If the relation between cue and target is strong—that is, the cue usually predicts the target location (say, 90 % of the time)—then participants learn to use the cue to predict the next target’s location. Sometimes, though, because the target may be presented at a location not indicated by the cue, the participant is misled, and we call this an invalid trial (Figure 7.15b). Finally, the researcher may include some cues that give no information about the most likely location of the impending target; we call this situation a neutral trial (Figure 7.15c).

In cuing studies of voluntary attention, the time between the presentation of the attention-directing cue and the presentation of the subsequent target might be very brief or last up to a second or more. When participants are not permitted to move their eyes to the cued spot, but the cue correctly predicts the target’s location, participants respond faster than when neutral cues are given (Figure 7.16). This faster response demonstrates the benefits of attention. In contrast, reaction times are slower when the stimulus appears at an unexpected location, revealing the costs of attention. If the participants are asked to discriminate some feature of the target, then benefits and costs of attention can be expressed in terms of accuracy instead of, or in addition to, reaction time measures.

Benefits and costs of attention have been attributed to the influence of covert attention on the efficiency of information processing. According to some theories, such effects result when the predictiveness of the cue induces the participants to direct their covert attention internally—a sort of mental “spotlight” of attention—to the cued visual field location. The spotlight is a metaphor to describe how the brain may attend to a spatial location. Because participants are typically required to keep their eyes on a central fixation spot on the viewing screen, internal or covert mechanisms must be at work.

Among others, University of Oregon professor Michael Posner and his colleagues (1980) have suggested that this attentional spotlight affected reaction times by influencing sensory and perceptual processing: Thus the stimuli that appeared in an attended location were processed faster than the stimuli that appeared in the unattended location. This enhancement of attended stimuli, a type of early selection, suggests that changes in perceptual processing can happen when the participant is attending a stimulus location. Now you might be thinking, “Ahhh, wait a minute there, fellas... responding more quickly to a target appearing at an attended location does not imply that the target was more efficiently processed in our visual cortex (early selection). These measures of reaction time—or behavioral measures more generally—are not measures of specific stages of neural processing. They provide only indirect measures. These time effects could solely reflect events going on in the motor system.” Exactly. Can we be sure that the perceptual system actually is responsible? In order to determine if changes in attention truly affected perceptual processing, researchers turned to some cognitive neuroscience methods in combination with the voluntary cuing paradigm.

Although most of the experiments discussed in this chapter focus on the visual system and, hence, on visual attention, this should not be taken to suggest that attention is only a visual phenomenon. Selective attention operates in all sensory modalities. In fact, it was investigations of the auditory system, spurred on by curiosity about the cocktail party effect, that led to the first round of cognitive neuroscience studies looking at the affect of attention on perceptual selection. These early studies made it clear that attention did affect early processing of perceptual stimuli, but not without some bumps in the road. Take a look at the How the Brain Works: Attention, Arousal, and Experimental Design box before we proceed to the land of visual attention.

After these early auditory ERP studies were conducted (heeding Näätänen’s precautions discussed in the box), vision researchers became interested in studying the effects of attention on their favorite sense. They wanted to know if attention affected visual processing, and if so, when and where during processing it occurred. We begin with research of voluntary visual-spatial attention. Visual spatial attention involves selecting a stimulus on the basis of its spatial location. It can be voluntary, such a when you look at this page, or it can be reflexive, when you might glance up having been diverted by a motion or flash of light.

Cortical Attention Effects Neural mechanisms of visual selective attention have been investigated using cuing paradigm methods, which we have just described. In a typical experiment, participants are given instructions to covertly (without diverting gaze from a central fixation spot) attend to stimuli presented at one location (e.g., right field) and ignore those presented at another (e.g., left field) while event-related potential (ERP) recordings are made (see Chapter 3, page 100).

Since the turn of the 19th century, scientists have known that the ascending auditory pathway carries two-way traffic. Each neural relay sends axons to the auditory cortex and also sends return axons back to the preceding processing stage, even out to the cochlea via the olivocochlear bundle (OCB). Because this appears to be a sign of top-down communication in the auditory system, researchers have investigated whether the behavioral effects of attention, like those revealed in dichotic listening studies, might be the result of gating that occurs very early in auditory processing, such as in the thalamus, brainstem, or even all the way back to the cochlea.

The esteemed Mexican neurophysiologist Raul Hernández-Peón and his colleagues (1956) attempted to determine whether phenomena like the cocktail party effect might result from a gating of auditory inputs in the ascending auditory pathways. They recorded the activity in neurons within the subcortical auditory pathway of a cat while it was passively listening to the sound from a speaker (Figure 1a). They compared those results with recordings from the same cat when it was ignoring the sound coming from the speaker. How did they know the cat was ignoring the sounds? They showed mice to the cat, thereby attracting its visual attention (Figure 1b). They found that the amplitude of activity of neurons in the cochlear nucleus was reduced when the animal attended to the mice—apparently strong evidence for early-selection theories of attention.

Unfortunately, these particular experiments suffered fatal flaws that could affect attention. The cat—being a cat—was more aroused once it spotted a mouse, and because a speaker was used to present the stimuli instead of little cat headphones, movements of the ears led to changes in the amplitudes of the signals between conditions. Hernández-Peón and his colleagues had failed to control for the differences either in the state of arousal or in the amplitude of the sound at the cat’s ears.

FIGURE 1 Early study of the neurophysiology of attention.
A sound was played to a cat through a loudspeaker under three conditions while recordings from the cochlear nucleus in the brainstem were obtained. (a) While the animal sits passively in the cage listening to sounds, the evoked response from the cochlear nucleus is robust. (b) The animal’s attention is attracted away from the sounds that it is hearing to visual objects of interest (a mouse in a jar). (c) The animal is once again resting and passively hearing sounds. The arrows indicate the responses of interest, and the horizontal lines indicate the onsets and offsets of the sounds from the loudspeaker.

These problems have two solutions, and both are necessary. One solution is to introduce experimental controls that match arousal between conditions of attention. The other is to carefully control the stimulus properties by rigorously monitoring ear, head, and eye positions.

In 1969, a Finnish psychologist, Risto Näätänen, laid out the theoretical issues that have to be addressed to permit a valid neurophysiological test of selective attention. Among the issues he noted were that the experimental design had to be able to distinguish between simple behavioral arousal (low state of arousal vs. high state of arousal) and truly selective attention (e.g., attending one source of relevant sensory input while simultaneously ignoring distracting events).

Indeed, when Hernández-Peón’s students repeated the 1956 experiment and carefully avoided changes in the sound amplitude at the ear, no differences were found subcortically between the neural response to attended and ignored sound.

The first physiological studies to control for both adjustments of peripheral sensory organs and nonspecific effects of behavioral arousal were conducted on humans by Steven Hillyard and his colleagues (1973) at the University of California, San Diego. ERPs were recorded because they provide a precise temporal record of underlying neural activity, and the ERP waves are related to different aspects of sensory, cognitive, and motor processing. Hillyard presented streams of sounds into headphones being worn by volunteers. Ten percent of the sounds were a deviant tone that differed in pitch. During one condition, participants were asked to attend to and count the number of higher pitched tones in one ear while ignoring those in the other (e.g., attend to right-ear sounds and ignore left-ear sounds). In a second condition, they were asked to pay attention to the stimuli in the other ear (e.g., attend to left-ear sounds and ignore right-ear sounds). In this way the researchers separately obtained auditory ERPs to stimuli entering one ear when input to that ear was attended and when it was ignored (while attending the other ear). The significant design feature of the experiment was that, during the two conditions of attention, the participants were always engaged in a difficult attention task, thus controlling for differing arousal states. All that varied was the direction of covert attention—to which ear the participants directed their attention. Figure 2 shows that the auditory sensory ERPs had a larger amplitude for attended stimuli, providing evidence that sensory processes were directed by attention. This result supported early-selection models and gives us a physiological basis for the cocktail party effect. Note that the subject also heard the sound through headphones to avoid the problem of differing sound strength at the ear drum, as occurred in the cat studies of Hernández-Peón.

FIGURE 2 Event-related potentials in a dichotic listening task.
The solid line represents the idealized average voltage response to an attended input over time; the dashed line, the response to an unattended input. Hillyard and colleagues found that the amplitude of the N1 component was enhanced when attending to the stimulus compared to ignoring the stimulus.

Looking at a typical ERP recording from a stimulus in one visual field (Figure 7.17), the first big ERP wave is a positive one that begins at 60–70 ms and peaks at about 100 ms (P1; first trough in Figure 7.17b) over the contralateral occipital cortex. It is followed by a negative wave that peaks at 180 ms (N1; Figure 7.17b). Modulations in the visual ERPs due to attention begin as early as 70–90 ms after stimulus onset, and thus, affect the P1 wave (Eason et al., 1969; Van Voorhis & Hillyard, 1977). When a visual stimulus appears at a location to which a subject is attending, the P1 is larger in amplitude than when the same stimulus appears at the same location but attention is focused elsewhere (Figure 7.17b). This is consistent with the attention affects observed in studies of auditory and tactile selective attention, which also modulates sensory responses.

This effect of visual attention primarily occurs with manipulations of spatial attention and not when attention is focused selectively on the features (e.g., one color vs. another) or object properties (e.g., car keys vs. wallet) of stimuli alone. Attention effects for the more complex tasks of feature attention or object attention are observed later in the ERPs (greater than 120-ms latency—but see Figure 7.38 and related text). We describe these effects later in the chapter when we discuss attention to stimulus features. Thus, it seems that spatial attention has the earliest effect on stimulus processing. This early influence of spatial attention may be possible because retinotopic mapping of the visual system means that the brain encodes space very early—as early as at the retina—and space is a strong defining feature of relevant versus irrelevant environmental events.

Where, within the visual sensory hierarchy, are these earliest effects of selective visuospatial attention taking place and what do they represent? The P1 attention effect has a latency of about 70 ms from stimulus onset, and it is sensitive to changes in physical stimulus parameters, such as location in the visual field and stimulus luminance. We’ve learned from intracranial recordings that the first volleys of afferent inputs into striate cortex (V1) take place with a latency longer than 35 ms, and that early visual cortical responses are in the same latency range as the P1 response. Taken together, these clues suggest that the P1 wave is a sensory wave generated by neural activity in the visual cortex, and therefore, its sensitivity to spatial attention supports early selection models of attention. We know from Chapter 3, however, that ERPs represent the summed electrical responses of tens of thousands of neurons, not single neurons. This combined response produces a large enough signal to propagate through the skull to be recorded on the human scalp. Can the effect of attention be detected in the response of single visual neurons in the cortex? For example, let’s say your attention wanders from the book and you look out your window to see if it is still cloudy and WHAT??? You jerk your head to the right to get a double take. A brand new red Maserati Spyder convertible is sitting in your driveway. As a good neuroscientist, you immediately think, “I wonder how my spatial attention, focused on this Maserati, is affecting my neurons in my visual cortex right now?” rather than “What the heck is a Maserati doing in my driveway?”

FIGURE 7.17 Stimulus display used to reveal physiological effects of sustained, spatial selective attention.
(a) The participant fixates the eyes on the central crosshairs while stimuli are flashed to the left (shown in figure) and right fields. (left panel) The participant is instructed to covertly attend to the left stimuli, and ignore those on the right. (right panel) The participant is instructed to ignore the left stimuli and attend to the right stimuli. Then the responses to the same physical stimuli, such as the white rectangle being flashed to left visual hemifield in the figure, are compared when they are attended and ignored. (b) Sensory ERPs recorded from a single right occipital scalp electrode in response to the left field stimulus. The waveform shows a series of characteristic positive and negative voltage deflections over time, called ERP components. Notice that the positive voltage is plotted downward. Their names reflect their voltage (P = positive; N = negative) and their order of appearance (e.g., 1 = first deflection). Attended stimuli (red trace) elicit ERPs with greater amplitude than do unattended stimuli (dashed blue trace).

Jeff Moran and Robert Desimone (1985) revealed the answer to this question (the former, not the latter). The scientists investigated how visuospatial selective attention affected the firing rates of individual neurons in the visual cortex of monkeys. Using single-cell recording, they first recorded and characterized the responses of single neurons in extrastriate visual area V4 (ventral stream area) to figure out what regions of the visual field they coded (receptive field location) and which specific stimulus features the neurons responded to most vigorously. The team found, for example, that neurons in V4 fired robustly in response to a single-colored, oriented bar stimulus (e.g., a red horizontal bar) more than another (e.g., a green vertical bar). Next, they simultaneously presented the preferred (red horizontal) and non-preferred (green vertical) stimuli near each other in space, so that both stimuli were within the region of the visual field that defined the neuron’s receptive field. Over a period of several months, the researchers had previously trained the monkeys to fixate on a central spot on a monitor, to covertly attend to the stimulus at one location in the visual field, and to perform a task related to it while ignoring the other stimulus. Responses of single neurons were recorded and compared under two conditions: when the monkey attended the preferred (red horizontal bar) stimulus, and when it instead attended the non-preferred (green vertical bar) stimulus that was located a short distance away. Because the two stimuli (attended and ignored) were positioned in different locations, the task can be characterized as a spatial attention task. How did attention affect the firing rate of the neurons?

When the red stimulus was attended, it elicited a stronger response (more action potentials fired per second) in the corresponding V4 neuron that preferred red horizontal bars than when the red stimulus was ignored while attending the green vertical bar positioned at another location. Thus, spatial selective attention affected the firing rates of V4 neurons (Figure 7.18). As with the ERPs in humans, the activity of single visual cortical neurons are found to be modulated by spatial attention.

Several studies have replicated the attention effects observed by Moran and Desimone in V4 and have extended this finding to other visual areas, including later stages of the ventral pathway in the inferotemporal region. In addition, work in dorsal-stream visual areas has demonstrated effects of attention in the motion processing areas MT and MST of the monkey. Researchers also investigated whether attention affected even earlier steps in visual processing—in primary visual cortex (V1), for example.

Carrie McAdams and Clay Reid (2005), at Harvard Medical School, carried out experiments to determine which level of processing within V1 was influenced by attention. Recall from Chapter 6 that many stages of neural processing take place within a visual area, and in V1 different neurons display characteristic receptive-field properties—some are called simple cells, others complex cells, and so on. Simple cells exhibit orientation tuning and respond to contrast borders (like those found along the edge of an object). Simple cells are also relatively early in the hierarchy of neural processing in V1—so, if attention were to affect them, this would be further evidence of how early in processing, and by what mechanism, spatial attention acts within V1.

McAdams and Reid trained monkeys to fixate on a central point and covertly attend a black-and-white flickering noise pattern in order to detect a small, colored pixel that could appear anywhere within the pattern (Figure 7.19a). When the monkeys detected the color, they were to signal this by making a rapid eye movement (a saccade) from fixation to the location on the screen that contained that color. The attended location would be positioned either over the receptive field of the V1 neuron they were recording or in the opposite visual field. Thus, the researchers could evaluate responses of the neuron when that region of space was attended and when it was ignored (in different blocks). They also could use the flickering noise pattern to create a spatiotemporal receptive-field map (Figure 7.19b) showing regions of the receptive field that were either excited or inhibited by light. In this way, the researchers could first determine whether the neuron had the properties of simple cells. They could also see whether attention affected the firing pattern and receptive-field organization. What did they come up with? They found that spatial attention enhanced the responses of the simple cells but did not affect the spatial or temporal organization of their receptive fields (Figure 7.19c).

FIGURE 7.18 Spatial attention modulates activity of V4 neurons.
The areas circled by dashed lines indicate the attended locations for each trial. A red bar is an effective sensory stimulus, and a green bar is an ineffective sensory stimulus for this neuron. The neuronal firing rates are shown to the right of each monkey head. The first burst of activity is to the cue, and the second burst in each image is to the target array. (a) When the animal attended to the red bar, the V4 neuron gave a good response. (b) When the animal attended to the green bar, a poor response was generated.

FIGURE 7.19 Attention effects in V1 simple cells.
(a) The stimulus sequence began with a fixation point and two color locations that would serve as saccade targets. Then two flickering black-and-white patches appeared, one over the neuron’s receptive field and the other in the opposite visual field. Before the onset of the stimuli, the monkey was instructed which of the two patches to attend. The monkey had been trained to covertly attend the indicated patch to detect a small color pixel that would signal where a subsequent saccade of the eyes was to be made (to the matching color) for a reward. (b) The spatiotemporal receptive field of the neuron when unattended (attend opposite visual field patch) and when attended. Each of the eight panels corresponds to the same spatial location as that of the black-and-white stimulus over the neuron’s receptive field. The excitatory (red) and inhibitory (blue) regions of the receptive field are evident; they are largest from 23.5 to 70 ms after stimulus onset (middle two panels). Note that the amplitudes of the responses were larger when attended than when unattended. This difference can be seen in these receptive-field maps and is summarized as plots in (c).

Does the same happen in humans? Yes, but different methods have to be used, since intracranial recordings are rarely done in humans. Neuroimaging studies of spatial attention show results consistent with those from cellular recordings in monkeys. Whole brain imaging studies have the advantage that one may investigate attention effects in multiple brain regions all in one experiment. Such studies have shown that spatial attention modulates the activity in multiple cortical visual areas. Hans-Jochen Heinze and his colleagues (1994) directly related ERP findings to functional brain neuroanatomy by combining positron emission tomography (PET) imaging with ERP recordings. They demonstrated that visuospatial attention results in modulation of blood flow (related to neuronal activity) in visual cortex. Subsequent studies using fMRI have permitted a more fine-grained analysis of the effects of spatial attention in humans.

For example, Joseph Hopfinger and his colleagues (2000) used a modified version of a spatial cuing task combined with event-related fMRI. On each trial, an arrow cue was presented at the center of the display and indicated the side to which participants should direct their attention. Eight seconds later, the bilateral target display (flickering black-and-white checkerboards) appeared for 500 ms. The participants’ task was to press a button if some of the checks were gray rather than white, but only if this target appeared on the cued side. The 8-s gap between the arrow and the target display allowed the slow hemodynamic responses (see Chapter 3) linked to the attention-directing cues to be analyzed separately from the hemodynamic responses linked to the detection of and response to the target displays. The results are shown in Figure 7.20 in coronal sections through the visual cortex of a single participant in the Hopfinger study. As you can see in this figure, attention to one visual hemifield activated multiple regions of visual cortex in the contralateral hemisphere.

FIGURE 7.20 Selective attention activates specific regions of the visual cortex, as demonstrated by event-related fMRI.
(a) Areas of activation in a single participant were overlaid onto a coronal section through the visual cortex obtained by structural MRI. The statistical contrasts reveal where attention to the left hemifield produced more activity than attention to the right (reddish to yellow colors, left) and the reverse, where attention to the right hemifield elicited more activity than did attention to the left (bluish colors, right). As demonstrated in prior studies, the effects of spatial attention were activations in the visual cortex contralateral to the attended hemifield. (b) The regions activated by attention (shown in black outline) were found to cross multiple early visual areas (shown as colored regions—refer to key).

Roger Tootell and Anders Dale at Massachusetts General Hospital (R. Tootell et al., 1998) investigated how all of the attention-related activations in visual cortex related to the multiple visual cortical areas in humans using retinotopic mapping. That is, they wanted to differentiate and identify one activated visual area from another on the scans. They combined high-resolution mapping of the borders of early visual cortical areas (retinotopic mapping; see Chapter 3) with a spatial attention task. Participants were required to selectively attend to stimuli located in one visual field quadrant while ignoring those in the other quadrants; different quadrants were attended to in different conditions while the participants’ brains were scanned with fMRI methods. This permitted the researchers to map the attentional activations onto the flattened computer maps of the visual cortex, thus permitting the attention effects to be related directly to the multiple visual areas of human visual cortex.

They found that spatial attention produced robust modulations of activity in multiple extrastriate visual areas, as well as a smaller modulation of striate cortex (V1; Figure 7.21). This work provides a high-resolution view of the functional anatomy of multiple areas of extrastriate and striate cortex during sustained spatial attention in human visual cortex.

Now we know that spatial attention does influence the processing of visual inputs. Attended stimuli produce greater neural responses than do ignored stimuli, and this difference is observed in multiple visual cortical areas. Is the effect of spatial attention different in the different visual areas? It seems so. The Tootell fMRI work hints at this possibility, because attention-related modulation of activity in V1 appeared to be less robust than that in extrastriate cortex; also, work by Motter (1993) suggested a similar pattern in the visual cortex of monkeys. If so, what mechanisms might explain a hierarchical organization of attention effects as you move up the visual hierarchy from V1 through extrastriate cortical areas?

FIGURE 7.21 Spatial attention produced robust modulation of activity in multiple extrastriate visual areas, as demonstrated by fMRI.
Panel a shows the retinotopic mappings of the left visual field for each participant, corresponding to the polar angles shown at right (which represents the left visual field). Panel b shows the attention-related modulations (attended versus unattended) of sensory responses to a target in the upper left quadrant (the quadrant to which attention was directed is shown at right in red). Panel c shows the same for stimuli in the lower left quadrant. In b and c, the yellow to red colors indicate areas where activity was greater when the stimulus was attended to than when ignored; the bluish colors represent the opposite, where the activity was greater when the stimulus was ignored than when attended. The attention effects in b and c can be compared to the pure sensory responses to the target bars when passively viewed (d).

Robert Desimone and John Duncan (1995) proposed a biased competition model for selective attention. Their model may help explain two questions. First, why are the effects of attention larger when multiple competing stimuli fall within a neuron’s receptive field? And second, how does attention operate at different levels of the visual hierarchy as neuronal receptive fields change their properties? In the biased competition model, when different stimuli in a visual scene fall within the receptive field of a visual neuron, the bottom-up signals from the two stimuli compete like two snarling dogs to control the neuron’s firing. The model suggests that attention can help resolve this competition in favor of the attended stimulus. Given that the sizes of neuronal receptive fields increase as you go higher in the visual hierarchy, there is a greater chance for competition between different stimuli within a neuron’s receptive field, and therefore, a greater need for attention to help resolve the competition (to read more about receptive fields, see Chapter 3).

Sabine Kastner and her colleagues (1998) used fMRI to investigate the biased competition model during spatial attention in humans (Figure 7.22). To do this, they first asked whether, in the absence of focused spatial attention, nearby stimuli could interfere with one another. The answer was yes. They found that when they presented two nearby stimuli simultaneously, the stimuli interfered with each other and the neural response evoked by each stimulus was reduced compared to when one stimulus was presented alone. If attention is introduced and directed to one stimulus in the display, however, then simultaneous presentation of the competing stimulus no longer interferes (Figure 7.23), and this effect tended to be larger in area V4 than in V1. The attention focused on one stimulus attenuates the influence of the competing stimulus. To return to our analogy, one of the snarling dogs (the competing stimulus) is muzzled.

It appears to be the case that, for a given stimulus, spatial attention operates differently at early (V1) versus later (e.g., V4) stages of the visual cortex. Why? Perhaps because the neuronal receptive fields differ in size from one visual cortical area to the next. Thus, although smaller stimuli might fall within a receptive field of a single V1 neuron, larger stimuli would not; but these larger stimuli would fall within the larger receptive field of a V4 neuron. In addition, exactly the same stimulus can occupy different spatial scales depending on its distance from the observer. For example, look at the flowers in Figure 7.24. When viewed at a greater distance (panel b), the same flowers occupy less of the visual field (compare what you see in the yellow circles). All of the flowers actually could fall into the receptive field of a single neuron at an earlier stage of the visual hierarchy. This observation suggests that attention operates at different stages of vision, depending on the spatial scale of the attended and ignored stimuli. Does it? How would you design a study to answer this question?

Max Hopf and colleagues (2006) combined recordings of ERPs, magnetoencephalography (MEG; see Chapter 3), and fMRI. The simple stimuli they used are shown in Figure 7.25a–c. In each trial, the target appeared as either a square or a group of squares, small or large, red or green, and shifted either up or down in the visual field. Participants were to attend to the targets of one color as instructed and to push one of two buttons depending on whether the targets were shifted up or down. The study revealed that attention acted at earlier levels of the visual system for the smaller targets than it did for the large targets (Figure 7.25d). So, although attention does act at multiple levels of the visual hierarchy, it also optimizes its action to match the spatial scale of the visual task.

Now that we have seen the effects of attention on the cortical stages of the visual hierarchy, have you started to wonder if attention might also cause changes in processing at the level of the subcortical visual relays? Well, others have also been curious, and this curiosity stretches back for more than 100 years. Recall the reflections of Helmholtz that we described earlier about the possible mechanisms of covert spatial attention? (See also How The Brain Works: Attention, Arousal, and Experimental Design.) Contemporary researchers have been able to shed light on this question of whether attention might influence subcortical processing.

Subcortical Attention Effects Could attentional filtering or selection occur even earlier along the visual processing pathways—in the thalamus or in the retina? Unlike the cochlea, the human retina contains no descending neural projections that could be used to modulate retinal activity by attention. But massive neuronal projections do extend from the visual cortex (layer 6 neurons) back to the thalamus. These projections synapse on neurons in what is known as the thalamic reticular nucleus (TRN; also known as the perigeniculate nucleus), which is the portion of the reticular nucleus that surrounds the lateral geniculate nucleus (LGN) (Figure 7.26).

These neurons maintain complex interconnections with neurons in the thalamic relays and could, in principle, modulate information flow from the thalamus to the cortex. Such a process has been shown to take place in cats during intermodal (visual–auditory) attention (Yingling & Skinner, 1976). The TRN was also implicated in a model to select the visual field location for the current spotlight of attention in perception—an idea proposed by Nobel laureate Francis Crick (1992). Is there support for such a mechanism?

Studies on monkeys in which attention affected the metabolic activity of the LGN neurons provided initial hints that attention might influence LGN processing (Vanduffel et al., 2000). Subsequent studies by Sabine Kastner and her colleagues used high-resolution fMRI to assess whether attention had the same influence in the human LGN (reviewed in Kastner et al., 2006). Researchers presented participants with a bilateral array of flickering checkerboard stimuli (Figure 7.27a), which activated the LGN and multiple visual cortical areas (Figure 7.27b). Participants were cued to attend to either the left or right half of the array. The results (Figure 7.27c) showed that the amplitude of the activation was greater in the LGN and visual cortex that were contralateral to the attended array compared to the activity in response to the unattended array. So, highly focused visuospatial attention can modulate activity in the thalamus. Since fMRI studies do not provide timing information, however, it is hard to know what such effects indicate. Do they reflect attentional gating of the afferent LGN neurons heading to V1? Or instead, do they reflect reafferent feedback to the thalamus from the cortex that is not the incoming afferent volley of information? McAlonan, Cavanaugh, and Wurtz (2008), at the National Eye Institute, recorded from LGN relay neurons and the surrounding TRN neurons of monkeys that had been trained to attend covertly to a target at one location while ignoring other targets. When the monkeys’ attention was directed to the location of the stimulus within the LGN neuron’s receptive field, the firing rate of the neuron increased (Figure 7.28a). In addition, however, the firing rate decreased in the surrounding TRN neurons (which, as you will recall, are not relay neurons, but instead are interneurons that receive input from the visual cortex; Figure 7.28b). Why is that? Well, we know from other work that the TRN neurons synapse onto the LGN neurons with inhibitory signals.

FIGURE 7.25 Study of effects of attention for different stimulus spatial scales.
(a) Example of an entire stimulus array from the spatial scale experiment. (b–c) Examples of the stimuli within a single quadrant of the array with the target at small (b) and large (c) scales. (d) MEG measures for the N2pc effect (an ERP reflecting focused attention) from 250 to 300 ms after the onset of the array, from a single volunteer. Large-scale trials (top rows) and small-scale trials (bottom rows) are shown on a ventral view (left panel images) and a left lateral view (right panel images) of the right hemisphere. One can see that for the small-scale trials, the activity in the brain is more posterior, reflecting neural responses from earlier stages of the visual system.

We can now explain the entire circuit. Attention involves either activating or inhibiting signal transmission from the LGN to visual cortex via the TRN circuitry. Either a descending neural signal from the cortex, or a separate signal from subcortical inputs travels to the TRN neurons. These inputs to the TRN can either excite the TRN neurons, thereby inhibiting information transmission from LGN to visual cortex, or the inputs can suppress the TRN neurons. Thus, transmission from LGN to visual cortex increases. The latter mechanism is consistent with the increased neuronal responses observed for the neurons in LGN and V1 when coding the location of an attended stimulus.

These studies demonstrate that highly focused spatial attention can modulate activity early in the visual system in the subcortical relay nuclei in the thalamus. This finding provides strong support for the early-selection models of attention. As you know, however, our attention is not always highly focused. In fact, yours may not be right now. You may have had to read the last couple of sentences over again. By passing these modulations along passively to higher visual areas, do these early modulations form the basis for all spatial attention effects in the visual system? Alternatively, can spatial attention act independently at multiple stages of visual processing (i.e., LGN, V1, and extrastriate cortical areas)? To learn more about this question, see How the Brain Works: Shocking Studies of Attention.

So far in our discussion on spatial attention, we have considered voluntary attention. We can voluntarily direct our attention to the words on this page or to remembering what we had for breakfast. Oftentimes, however, things in the environment attract our attention without our cooperation. This is known as reflexive attention, and it is activated by stimuli that are conspicuous in some way. The more salient (conspicuous) the stimulus, the more easily our attention is captured: Think of how we respond to a rapid movement at the corner of our eye (eek! a rat!), the shattering of glass in a restaurant, or someone whistling as they walk by the open door of the lecture hall. Heads turn toward the sounds and sights and then wag back a moment or two later, unless the event is behaviorally relevant. This head wagging may happen before we can prevent it, because our reflexive attention may lead to overt orientation to the sensory stimulus—overt because heads and eyes turn toward the event. Even without overt signs of orientation, however, covert attention can be attracted to sensory events. This leads to a question: Are reflexive and voluntary attention processed in the same way? To tackle this question, we can use a variant of the cuing method (see Figure 7.15) to demonstrate this phenomenon experimentally.

FIGURE 7.27 Functional MRI study of spatial attention effects in the lateral geniculate nucleus (LGN).
(a) Before stimulus onset, an arrow cue at fixation instructed the participants which hemifield to attend. Next, a checkerboard stimuli presented bilaterally for 18 s (shown as blue shaded area in c). The task was to detect randomly occurring luminance changes in the flickering checks in the cued hemifield. (b) Functional MRI activations (increased BOLD responses) were observed in the LGN (red box) and in multiple visual cortical areas (green box). (c) Increased activations were seen when the stimulus in the hemifield contralateral to the brain region being measured was attended. The effect was observed both in the LGN (top) and in multiple visual cortical areas (bottom).

The effects of reflexive attention can be demonstrated by examining how a task-irrelevant flash of light somewhere in the visual field affects the speed of responses to subsequent task-relevant target stimuli. This method is referred to as reflexive cuing or exogenous cuing, because attention is controlled by low-level features of an external stimuli, not by internal voluntary control. Although the light flash “cues” do not predict the location of subsequent targets, responses are faster to targets that appear in the vicinity of the light flash—but only for a short time after the flash, about 50–200 ms. These types of effects tend to be spatially specific. That is, they influence processing in and around the location of the reflexive cue only. Therefore, they can also be described by the spotlight metaphor introduced earlier in this chapter. In this case, however, the spotlight is reflexively attracted to a location and is short-lived.

The interesting thing is that when more than about 300 ms passes between the task-irrelevant light flash and the target, the pattern of effects on reaction time is reversed. Participants respond more slowly to stimuli that appear in the vicinity of where the flash had been. This phenomenon is called the inhibitory aftereffect or, more commonly, inhibition of return (IOR). Why would reflexive attentional orienting have profound variations in its effect over time following a sensory event? Consider the advantages of this kind of system. If sensory events in the environment caused reflexive orienting that lasted for many seconds, people would be continually distracted by things happening around them and would be unable to attend to a goal. Our ancestors might never have made it to reproductive age and thus, we wouldn’t be here reading this book. They would have been watching for a lion or looking for food, but then been distracted and entranced by a bird’s song—whoops, missed the lion! Or whoops, no meal, again! In today’s world, imagine the consequences if a driver’s attention became reflexively focused on a distraction off to the side of the road and then remained focused on that event for more than an instant. Our automatic orienting system has built-in mechanisms to prevent reflexively directed attention from becoming stuck at a location for more than a couple of hundred milliseconds. The reflexive capturing of attention subsides, and the likelihood that our attention will be drawn back to that location is reduced slightly. Does this mean that things that attract our attention reflexively cannot be attended for longer than a couple of hundred milliseconds? No, we know from experience that isn’t true. If the event is important and salient, we can rapidly invoke our voluntary mechanisms to sustain attention longer, thereby overriding the inhibition of return. Thus, the nervous system has evolved clever, complementary mechanisms to control attention so that we can function in a cluttered, rapidly changing sensory world.

It may seem that there is not much difference between the responses to an endogenous cue and an exogenous cue. Both result in attention shifts that enhance the processing of attended sensory stimuli and decrease that of the unattended. In the case of reflexive attention, however, the cuing effect is quick and short-lived, and processing of stimuli in the neighborhood of the cue is enhanced. With voluntary attention cuing, however, the effect is slower and more sustained. Do these differences in processing represent different neural mechanisms?

We have learned that voluntarily focusing attention at a location in response to verbal instructions or instructive visual pre-cues will enhance the visual responses to stimuli occurring at that location. Do these same changes occur when our attention is reflexively attracted to a location in the visual field by a sensory event? Joseph Hopfinger and colleagues (1998, 2001) answered yes to this question. They recorded ERPs in response to target stimuli in a reflexive cuing task like the one described earlier (Figure 7.29a). They found that the early occipital P1 wave is larger for targets that quickly follow a sensory cue at the same location versus trials in which the sensory cue and target occur at different locations. As the time after cuing grows longer, however, this effect reverses and the P1 response diminishes—and may even be inhibited—just as in measurements of reaction time (Figure 7.29b). Therefore, these data indicate that both reflexive (stimulus driven) and voluntary (goal directed) shifts in spatial attention induce similar physiological modulations in early visual processing. Presumably, the neural networks implementing these attentional modulations of sensory analysis are different, reflecting the differing ways in which attentional control is triggered for the two forms of attention.

In everyday perception, voluntary attention (driven by our goals) and reflexive attention (driven by stimuli in the world) interact in a push-pull fashion, struggling to control the focus of our attention. For example, we frequently search about for a specific item in a cluttered scene. Perhaps we watch for a friend coming out of the building after class, or we look for our suitcase on the baggage claim carousel of a busy airport. If the suitcase is red and covered with flowered stickers, the search is quite easy. If the suitcase is a medium-sized black bag with rollers, the task can be quite challenging. As you cast your gaze around for that friend or suitcase, you don’t keep going back to places that you have just scanned. Instead, you are biased, moving your eyes to new objects in new locations. The last time you stood in baggage claim, you probably didn’t wonder what role attentional processes play in this visual search process. Are you getting curious now? How are voluntary and reflexive spatial attention mechanisms related to visual search?

It is clear that effects of visual attention, particularly spatial attention, can be detected at multiple stages of visual information processing. The effects begin as early as the LGN of the thalamus, and they include early and later stages of visual cortical processing. It is also clear that information processing in visual cortex is influenced by top-down attentional control systems that bias the activity of visual neurons. One question that remains unclear is whether attention can influence visual information processing at multiple loci along the ascending visual pathways, or if instead, attentional filtering takes place by influencing a single early stage of processing, such as the subcortical relays, and then passively transmits the effects of attention to later stages of visual analysis.

Functional imaging studies demonstrate that, when participants prepare for a target in one location in the visual field while ignoring other locations, the background (pre-target) activity in multiple loci in the ascending visual pathways increases, suggesting that attention does act simultaneously at multiple stages. But this evidence is somewhat indirect. Bestmann and colleagues (2007) attained more direct evidence with transcranial magnetic stimulation (TMS) to demonstrate that spatial attention can act directly on sites in the cortex. Using TMS in human volunteers, they induced phosphenes (see Chapter 5, page 211) by the direct stimulation of visual cortex and were able to demonstrate that visual percepts were influenced by attention.

They conducted this study as follows (Figure 1). Attention was covertly (without displacement of gaze) directed toward a particular location (left or right) during a task involving real visual stimuli. In two different conditions, attention was either cued with a predictive arrow cue or directed to the left or right in a sustained manner throughout a block consisting of many trials (essentially versions of the designs discussed previously in this chapter). The trick here was that, on some trials, instead of a real visual stimulus, TMS was applied to produce a phosphene, either at the attended location or in the opposite (unattended) hemifield. Thus, by measuring the phosphene threshold (PT), which is the amplitude of the TMS pulse needed to create a phosphene for the observer, the researchers could determine whether perception of the TMS-induced phosphenes was influenced by spatial attention.

Bestmann and colleagues found that PTs were lowered for trials in which the TMS pulse was delivered to the visual cortex that corresponded to the attended (contralateral) visual hemifield: This meant that spatial attention was modulating the TMS-induced phosphenes. Because these “signals” in visual cortex did not pass through the thalamic relays to reach visual cortex, this evidence suggests that attention can act directly within the sensory cortex and does not rely on the modulation of visual inputs in the LGN. That is, the direct activation of visual cortex using transcranial magnetic stimulation in humans provides converging evidence that attentional control is not limited to gating subcortical inputs to cortex. This statement does not, of course, imply that attention does not influence processing in the LGN (in this chapter, we have reviewed evidence that this influence does occur). Rather, the TMS study demonstrates that attention can act directly on cortical processing, independently of its actions in subcortical structures.

FIGURE 1 Attentional modulations of phosphenes induced by transcranial magnetic stimulation (TMS) in humans.
(a–d) Stimuli and task sequence in the spatial cuing design. Participants were shown an arrow cuing them to the left or right visual field. After a delay of 500 to 1,000 ms, they were shown either a target (a) or TMS-induced phosphenes (b) at the attended location, or a target (c) or TMS-induced phosphenes (d) at the unattended location. (e) Cuing produced the well-known reaction time benefits on target processing: Targets at the attended location were detected faster. (f) Similarly, the phosphene threshold (PT) was lower for TMS-induced phosphenes at the attended location.

While we don’t know if Anne Treisman (Princeton University) had suitcase issues at the airport, we do know that she and her colleagues have long been curious about the mechanisms of visual search. In one set of experiments, they observed that targets are located more quickly among a field of distracters if the target can be identified by a single stimulus feature, such as color (e.g., a red O among green Xs and Os). It doesn’t matter how many distracters appear in the array. We can demonstrate this relation by plotting participants’ reaction times as a function of the number of distracter items in the display (search function), as shown in Figure 7.30. When the target can be identified by a single feature, such as the red O in Figure 7.30a (or one red suitcase in a sea of black suitcases), the resulting search function is flat (Figure 7.30c, blue line). We refer to this phenomenon as pop-out because the red O literally appears to pop out of the array of green letters based on its color alone. If the target shares features with the distracters, however, so that it cannot be distinguished by a single feature (e.g., a red O among green Xs and Os and red Xs, as in Figure 7.30b, or a medium-sized red suitcase among medium-sized black suitcases and large red and black suitcases), then the time it takes to determine whether the target is present or absent in the array increases with the number of distracters in the array. The resulting search function is a sloped line (Figure 7.30c, red line). This type of search is known as a conjunction search because the target is defined by the conjunction of two or more features (e.g., the color red and the letter’s identity as an O, or the color and size of the suitcase).

To explain why conjunction targets take longer to find, Treisman and Gelade (1980) proposed that while elementary stimulus features such as color, motion, shape, and spatial frequency can be analyzed preattentively and in parallel within multiple specialized feature maps (located within visual cortical areas), spatial attention is more complicated. Spatial attention must be directed to relevant stimuli in order to integrate the features into the perceived object, and it must be deployed in a sequential manner for each item in the array. This condition is necessary to link the information (in this case, color and letter identity, or suitcase color and size) in the different feature maps so that the target can be analyzed and identified. This concept is called the feature integration theory of attention. Returning to the spotlight analogy, the idea here is that a spotlight of attention must move sequentially from one item in the array to another. Does this theory relate to the metaphorical spotlight of attention we introduced during our discussion of voluntary and reflexive spatial attention? Some evidence suggests that it does indeed.

To test this idea, Jeremy Wolfe and his colleagues (2000) at Harvard University asked: How does voluntary spatial attention modulate visual search? They employed a visual search task in which two conditions were compared. In one condition, participants knew in advance where to focus their attention, but in the other, the participants were not instructed where to focus their attention. Under the former condition—in which participants knew in advance where the target might be—the participants could use voluntary attention to perform the task. They found that people took longer to find their targets when deliberate movements of attention were required in a visual search task compared to when deliberate movements of attention were not required and search was permitted to proceed automatically. These results may seem odd, but what they tell us is that visual search is most rapid when you permit the focus of attention to be driven by the visual sensory information in the array, rather than by executing a slow, voluntarily controlled search of the items. In other words, the brain automatically scans the visual world with a fast, automatic spotlight of attention. You will find that red suitcase faster if you allow your attention to wander randomly rather than directing your search from one suitcase to the next in an orderly way. We still don’t know, however, how that automatic spotlight moves. Wolfe and colleagues conjectured that it also moved sequentially from item to item, but there is another possibility.

In most models, this automatic process involves low-level feature maps (maps about such things as borders, line orientation, color, etc.) of the visual world that provide information about the salience of objects. Based on feature information, spatial attention is reflexively biased toward the locations of the most salient objects. Then, the spotlight of attention can be focused on the location of interest, linking the features of the item, and enabling discrimination and identification in order to determine if it is the item of interest (There’s my suitcase!). This model suggests that the spotlight of attention in visual search might be similar to the spotlight of attention observed in cuing paradigms. Is there any way to determine if there is a relationship between the spotlight of attention demonstrated in physiological studies (e.g., see Figure 7.17) and the findings from visual search studies?

Steven Luck and his colleagues (1993) hypothesized that if a probe stimulus were to appear at a location where spatial attention is focused during visual search, then it would elicit larger visual ERPs than when a probe appeared at locations where attention was not focused. To test this hypothesis, participants were presented with arrays that contained a field of upright and inverted blue, green, and red “t” shapes. In each trial, the target they were to look for was a “t” of a particular color (blue or green), of which there would be only one that varied in location from trial to trial—a pop-out (see Figure 7.31a). At brief time intervals after the search array was presented, a solitary ERP-eliciting probe stimulus was flashed either at the location of the pop-out target item (to the “t” where attention had been drawn) or at the location of a distracter item on the opposite side of the array. The probe stimulus was the white outline of a square, which appeared around either the blue “t” or the green “t,” but never around a red “t.”

The probe elicited larger early visual responses (P1) at the location of the designated conjunction target (where their attention was focused) as compared to regions where only distracters were present, thus supporting Luck’s hypothesis. Perhaps a similar neural mechanism is at work for the early selection of visual information during visual search, as well as during voluntary attention in cuing and sustained attention paradigms. Of course, the difference is that, during visual search, the location of the target is not known until the search concludes. In the cuing paradigms or the sustained attention paradigms, however, attention is directed to a known location based on the information in the cue or in the verbal instructions given to the participant. Surprisingly, this difference doesn’t seem to matter. In both cases, spatial attention changes early processing in the visual cortex through neural mechanisms that appear to be quite similar.

FIGURE 7.31
(a) Stimuli were shown to participants, who were told to search for either a blue or green “t” on each trial, and to indicate with a button push whether that item was upright or inverted. The red “t’s” were always irrelevant distracters. An irrelevant white outlined square was flashed (50 ms duration) as a probe stimulus either around the blue or green “t.” Moreover, the white probe could be flashed around the blue or green item when the colored item was the target, or when it was merely an irrelevant distracter. In this way, the amplitude of the probe ERP could be taken as an index of the location and strength of spatial attention just after the onset of the search array, at the point where participants would have located the target and discriminated its form (upright or inverted). The white probe was flashed either 250 ms or 400 ms after the onset search array. The search array remained on the screen for 700 ms. (b) The irrelevant white probe elicited a larger sensory-evoked occipital P1 wave when it occurred at the location of a relevant target (e.g., blue “t”) compared to the irrelevant target (e.g., green “t”). These findings support the idea that focal spatial attention is directed to the location of the target in the array during visual search, and show that this corresponds to amplitude modulations in early visual cortex, just as in spatial cuing paradigms.

Despite knowing that spatial attention affects early processing in visual search, we still haven’t answered the question about how spatial attention arrived at the location of the conjunction target. Was spatial attention automatically moving freely from item to item until the target was located, as suggested by the work of Jeremy Wolfe and colleagues? Or was visual information in the array helping to guide the movements of spatial attention among the array items, as other models have proposed? That is, does spatial attention have to precede feature or object attention in a hierarchical fashion? Or can object features (e.g., shape and color) be identified and selected, at least to some extent, independently of spatial attention, as suggested in Treisman’s feature integration theory? Perhaps feature attention provides a signal that enables spatial attention to be directed to the location of a stimulus containing a relevant feature, whereupon more detailed analysis within the focus of spatial attention can take place. The neurophysiological evidence described next supports the latter schema, which has been predicted in numerous models (e.g., see A. Cohen & Ivry, 1989).

So far, we have focused on visual spatial attention, the ability to direct our attention to some locations at the expense of others in the environment. Although we have been concentrating on visual attention, for completeness we will add that spatial attention also influences auditory and somatosensory information processing. As our own experience tells us, we have learned that selectively attending to spatial locations, either voluntarily or reflexively, leads to changes in our ability to detect and respond to stimuli in the sensory world. As Robert Louis Stevenson pointed out, however, the world is full of objects of interest, some more interesting than others. For instance, when you gaze across the expanse of Monument Valley (Figure 7.32), your attention is not drawn to some random bush, but to the mesas. Why does that happen?

Objects are defined by their collection of elementary features, as we discussed in Chapters 5 and 6. We will now revisit these concepts with selective attention in mind.

How does selectively attending to a stimulus feature (e.g., motion, color, shape) or object properties (e.g., a face vs. a house) influence information processing? For instance, if cued to expect that an upcoming stimulus is moving, are we better able to discriminate the target stimulus if indeed it is moving than if it is unexpectedly not moving? If your friend says she will pick you up at the airport and will drive around the airport terminals until you spot her, will it take you longer to spot her if she is parked at the curb instead? And, of course, we still want to know how feature and spatial attention interact, given that the world is full of features and objects located in specific locations.

Marissa Carrasco and her colleagues at New York University performed a set of experiments to address these questions. They compared spatial attention and feature attention in a voluntary cuing paradigm. The dependent measure of attention was detection accuracy (Liu et al., 2007). In one condition (using spatial attention), arrow cues were used to indicate the location where attention should be directed. In the other condition (the feature attention condition), arrows indicated the direction of motion of the upcoming target (Figure 7.33a). The researchers found that prior knowledge from the cue produced the typical voluntary cuing effect for spatial attention: Participants were more accurate at detecting the presence of the target (a change in the velocity of moving dots) at the cued location compared to when the cue (a double-headed arrow) did not signal one location over another (Figure 7.33b, red line). In a similar vein, they found that, during the feature attention condition, cuing the direction of motion of the target also enhanced accuracy independently of whether it appeared in the left or right visual field array (Figure 7.33b, blue line). Thus, pre-cuing attention to a visual feature (motion direction in this case) improved performance. This finding tells us that attention can be directed in advance to spatial locations as well as to nonspatial features of the target stimuli. Now let’s ferret out the neural bases of selective attention to features and objects, and contrast these mechanisms with those of spatial attention.

In the early 1980s, Thomas Münte, a German neurologist working in Steve Hillyard’s lab developed a clever experimental paradigm (Hillyard & Münte, 1984). Using ERPs, they isolated the brain responses that are related to selectively attending the color of a stimulus from those related to attending stimulus location. Rather than cuing participants to different stimulus features, they presented participants with blocks of many trials where small red and blue vertical rectangles were flashed in a random sequence in the left and right visual fields (the rectangles could be tall or short). Each block of trials lasted a minute or so. Participants fixated on the central crosshairs on the screen while covertly attending to one color at the attended location. They ignored the other color at the attended location, as well as ignoring both colors at the unattended location.

For example, participants were told, “For the next minute, attend and push the button to the shorter red bars on the right only.” On the next block, they were told, “For the next minute, attend and push the button to the shorter blue bars on the right only.” In other blocks of trials, they were also told the same for the bars on the left. Thus, there were four different attention conditions, and the investigators could compare the ERPs generated under the four conditions. In this ingenious setup, the comparisons independently revealed the processing for spatial attention and feature attention. For example, spatial attention to a left–red stimulus (attend left vs. attend right) could be experimentally uncoupled from feature attention (attend red vs. attend blue). The brain responses for each of these conditions are shown in Figure 7.34. In Figure 7.34a, the ERPs show the typical spatial attention effects shown earlier in Figure 7.17 (solid vs. dotted ERP trace). Figure 7.34b shows the ERPs showing the color attention ERPs. Note the very different patterns that spatial and color attention produced in the ERPs, which are especially obvious in the ERP attention difference waves (Figure 7.34c vs. d). The early P1 wave that indexes spatial attention (top row) is absent for color attention (bottom row), which shows only longer latency changes in the waveform. Also of interest from this work is that effects of color attention were largely absent at the unattended location (lower right traces solid vs. dotted). This research indicates that both spatial and feature attention can produce selective processing of visual stimuli, and that the mechanisms for spatial and feature attention differ. Good to know, but exactly where do these feature attention effects take place in the brain?

Well, it depends. Maurizio Corbetta and his colleagues at Washington University investigated what neural systems are involved in feature discrimination under two different conditions: divided attention and selective attention (Corbetta et al., 1991). In one of the first neuroimaging studies of selective attention, the researchers used PET to identify changes that occur in extrastriate cortex and elsewhere, when people selectively attend to a single stimulus feature such as color, shape, or motion versus when their attention was divided among all three features (as a comparison condition). Radioactive water was used as a tracer to monitor blood flow in the brain, as volunteers were shown pairs of visual displays containing arrays of stimulus elements. The first display of each trial was a reference stimulus, such as a red square; the second was a test stimulus, perhaps a green circle. The participants’ task during the selective attention condition was to compare the two arrays to determine whether a change had occurred to a pre-specified stimulus dimension (color, shape, or motion). During the divided attention condition, participants were instructed to detect a change in any of the three stimulus dimensions. This experimental design permitted the investigators to contrast brain activity under conditions in which the participants selectively attended a particular stimulus dimension (e.g., only color) with the condition in which they divided their attention among all stimulus dimensions. As you might expect, behavioral sensitivity for discriminating slight changes in a stimulus was higher when judging only one feature (selective attention) rather than multiple features (divided attention).

Selective attention to one feature activated distinct, largely nonoverlapping regions of extrastriate cortex (Figure 7.35) in comparison to divided attention. Extrastriate cortical regions specialized for the perceptual processing of color, form, or motion were modulated only during visual attention to the corresponding stimulus features. These findings provide additional support for the idea that selective attention, in modality-specific cortical areas, alters the perceptual processing of inputs before the completion of feature analysis. Subsequent fMRI studies have identified specialized areas of human visual cortex that process features, such as stimulus motion or color. Corresponding areas had been found previously in monkey visual cortex. These specialized feature analysis regions are modulated by selective visual attention, as suggested by the earlier work of Corbetta and colleagues.

When do these various attention effects occur during processing? To address this question, one study combined MEG and fMRI in order to provide temporal and spatial information (Schoenfeld et al., 2007). Participants were cued to attend selectively to either changes in color or changes in motion that could occur in an upcoming display (Figure 7.36a). The stimulus sequence randomly presented motion and color changes, permitting the measurement of brain activity in response to changes in either feature as a function of attention to motion or color. By using fMRI to localize brain regions sensitive to selective attention to color or motion, the investigators found (as expected) that attending to motion modulated activity in the visual cortical motion processing area MT/V5 (in the dorsal stream). Similarly, attending to color led to modulations in ventral visual cortex area V4 (in the ventral stream; Figure 7.36b–d). Importantly, the team’s related MEG recordings demonstrated that attention-related activity in these areas appeared with a latency of 100 ms or less after onset of the change in the stimulus—much sooner than previous studies had reported.

Thus, feature-based selective attention acts at relatively early stages of visual cortical processing with relatively short latencies after stimulus onset. Spatial attention, however, still beats the clock and has an earlier effect. We see, once again, that the effects of feature attention occur with longer latencies (100 ms vs. 70 ms after stimulus onset) and at later stages of the visual hierarchy (extrastriate cortex rather than striate cortex or the subcortical visual relays in the thalamus).

Are features selected before spatial attention is focused on a target location or after? Max Hopf and his colleagues (Hopf et al., 2004) used a visual search task while they recorded ERPs of participants to address this question. Before looking at this study, we need to talk about ERPs. Early P1 attention effects are followed in time by other ERPs that index nonspatial, feature-based attention, collectively referred to as feature selection ERPs (see Figure 7.34). Steve Luck and Steve Hillyard (1994) identified a human brain wave they called N2pc, where N2 refers to the second negative deflection, and pc refers to posterior electrode site contralateral to the attended stimulus, the location where the component appears. Experimental work has established that the N2pc component of a spatial selection ERP is a sign of the covert focusing of visual spatial attention during visual search. It represents a stage of processing that occurs before object recognition is completed.

In the study by Hopf and colleagues, the team investigated mechanisms of visual search by using feature selection ERPs and the N2pc as indices of feature and spatial attention, respectively. In the visual search task, the spatial distribution of distracters (variously colored and oriented C-shaped items) was varied independently of the location of the target. Participants could locate the target item by relying solely on its unique color (pop-out). The distracting features provided no information about the target’s location or identity. By using simultaneous ERP and MEG recordings, the researchers found that, 140 ms after onset of the search array, a feature selection ERP was generated in ventral occipitotemporal cortex (blue and red dots in Figure 7.37). This feature attention effect was quickly followed (about 30 ms later) by an N2pc response generated in more anterior regions of the occipitotemporal cortex, indicating that the participants were focusing spatial attention on the target.

These findings clearly demonstrate that feature selective attention may precede visuospatial attention when the location of the target is not known in advance (as is always the case in visual search paradigms). These intriguing results suggest that feature selection may guide subsequent shifts of attention to the locations of those features so that higher resolution mechanisms can process and identify conjunction targets. This concept would be consistent with the tenets of the feature integration theory, described earlier (see Figure 7.30 and associated text). If it is true that feature attention is separate from and does not depend on spatial attention, then we might expect to see effects of attending to a feature, such as color, outside the region of space that is currently attended. Although this expectation makes sense, it presents a quandary. Previous studies that investigated feature attention (color) at attended versus unattended locations found no evidence for feature attention outside the attended location (see Figure 7.34). What was going on?

Weiwei Zhang and Steve Luck (2009) reasoned that these previous studies had neglected to consider that attention selection is dependent on competition. After all, visual search involves multiple competitive stimuli—we’ve got all those other suitcases on the same baggage carousel where our suitcase should be. The researchers hypothesized that under such conditions, if attention to features can affect sensory processing outside the attended region of space, then when attending a feature, such as the color red because our suitcase is red, a task-irrelevant red item (a heart-shaped box of Valentine candy) presented in an unattended location (sitting in someone’s baggage cart) might elicit a larger response in visual cortex than a green item, which, in this case, is an irrelevant color. How did they test this hypothesis? They asked participants to view a monitor that displayed a continuous stream of red and green dots in one visual field (Figure 7.38). The participants were instructed to attend to the red dots but ignore the green dots. Sometimes the streams of red and green colored dots were presented simultaneously, and at other times the red and green streams of dots were presented sequentially. The participants’ task was to push a button when the brightness of the attended color stimuli decreased momentarily. Occasionally, as shown in the figure, a display of colored dots (probe stimulus) in either the attended or unattended color was flashed briefly to the opposite (unattended) side of the visual field. These stimuli were task-irrelevant, and to be ignored. Then, using recordings of ERPs from scalp electrodes, the researchers measured the activity in visual cortex to the probe, which was either the same color as the participant was attending, or a different color. They did these measurements in the two conditions of simultaneous and sequential presentation of the task-relevant red and green colored arrays. What did the scientists find?

FIGURE 7.38 Feature attention mechanisms with and without feature competition.
(a) In the condition where there was feature competition, red and green dots were intermingled as streams of stimuli during each 15 second trial. The participants’ task was to covertly attend to and detect a decrement in luminance of the to-be-attended color, while ignoring the intermingled dots of the other color. Task-irrelevant probe stimuli, either all-green or all-red occasionally flashed in the opposite visual hemifield. Thus, the probe stimuli could share the feature color with the attended stimuli or not. In a different condition where there was no feature competition (not shown), the task–relevant stimulus streams of red and green were presented separately in alternating sequence (i.e., all red or all green) with the task being the same one of detecting luminance decrements in one color while ignore the other. Once again in this condition, irrelevant probes of all red or all green dots were flashed in the unattended hemifield. (b) ERPs to the probe stimuli during feature competition. The ERPs showed a significant increase in amplitude to the irrelevant probe of the attended color compared to the irrelevant probe of the unattended color in the latency range of 80–130 msec over contralateral occipital scalp (characteristic of the P1 component). (c) ERPs to the probe stimuli when there was no feature competition (the red and green streams of dots were not present simultaneously). During the same short-latency time period (80–130 ms), there were no significant differences in the waveforms evoked by the irrelevant probes when they shared versus did not share the color of the attended targets in the streams. Feature attention, therefore, may only result in facilitation of relevant feature information outside the focus of spatial attention when there is competition between relevant and irrelevant features.

When the attended array contained both the attended (red) and unattended (green) dots at the same time—that is, when there was some stimulus competition—then the ERPs elicited by the probe were greater in amplitude for the attended color (Figure 7.38b). This was true even though the probe was flashed at an unattended location. Thus, attending to a color (red) in one stimulus location facilitated processing of stimuli in that same color (red) located at another location in the visual field that was outside the focus of spatial attention. Not only that, but the effect could occur at short latencies in the brain response—as short as attention effects for spatial attention are often observed (by 80–100 msec after probe onset). As we described earlier, spatial attention effects are typically found to precede nonspatial (feature and object) effects of attention (see Figure 7.34 for comparison), but not in the face of feature competition.

Importantly, the researchers found this effect only when the attended array contained both the attended (red) and unattended (green) color dots intermingled at the same time, not when they were presented sequentially (Figure 7.38c). Once again, we see how the degree of competition among stimuli can influence attention. This study provides evidence that attention to color may activate color-sensitive neurons across the visual field, and it can explain how searching for a red stimulus (or a red suitcase) may guide the focusing of spatial attention. That is, if the color red, for example, is the relevant feature, and it evokes a larger sensory response wherever in space it is located, this signal might summon spatial attention to that location.

We have now described the effects of spatial-based attention and feature-based attention in visual cortex. Can attention also act on higher order stimulus representations, namely, objects? When searching for a friend in a crowd, we don’t merely search where we think our friend will be, especially if we haven’t agreed on a place to meet. We also don’t search for our friend only by hair color (unless it is highly salient, like fluorescent pink). Rather, we look for the conjunction of features that define the person. For lack of a better word, we can refer to this quality as object properties—the collection of elementary stimulus features that, when combined in a particular way, yield an identifiable object or person. Behavioral work has demonstrated evidence for object-based attention mechanisms.

In a seminal study, John Duncan (1984) contrasted attention to location (spatial attention) with attention to objects (object-based attention). Holding spatial distance constant, he discovered that two perceptual judgments concerning the same object can be made simultaneously without loss of accuracy, whereas the same two judgments about different objects cannot. For instance, in a split second you can process that a dog is big and brown; but when two dogs are present, processing that one is big and the other is brown takes longer. This processing limitation in attending to two objects implicates an object-based attention system in addition to a space-based system. In line with this view, the behavioral reaction time costs (slowing) and benefits (speeding) of the spatial cues of attention are greater between two objects as compared to within one object (Egly et al., 1994). This result suggests that the spread of attention is facilitated within the confines of an object, or that there is an additional cost to move attention between objects, or both.

Notger Mueller and Andreas Kleinschmidt (2003) designed an fMRI study to determine what effect objects had on spatial attention. They wondered if attending to an object had any impact on processing in the early visual processing areas, and if so, what? They cued participants on a trial-by-trial basis to expect a target at one location in the visual field (e.g., upper left quadrant) and then presented targets there on most trials (valid trials). In a minority of trials, they presented them to uncued locations (invalid trials). Following the design of Egly et al. (1994), Mueller and Kleinschmidt included objects on the screen so that the uncued target could fall within the same object that was cued (but at another location in the object), or at another location that was not within the bounds of that object. To get a better idea of their design, look at Figure 7.39a.

The displayed objects were wrench-like figures, and these figures could be oriented horizontally on the screen or vertically. For example, when the wrenches were oriented horizontally and the upper left quadrant location was cued, the upper right quadrant location would be spatially uncued (unattended) but be within the same object. When the wrenches were vertically oriented, however, that location would be spatially uncued and within a different object. Mueller and Kleinschmidt replicated the behavioral reaction time effects of Egly and colleagues (Figure 7.39b). What’s more, they found that in visual cortical areas V1 through V4, increased activity occurred in uncued locations that were located on the same object (the wrench) as the cued location compared to when the uncued location was not on the same object that was cued (Figure 7.39c–d).

This result is evidence that the presence of objects influences the way spatial attention is allocated in space: In essence, attention spreads within the object, thereby leading to some activity for uncued locations on the object as well. An effect of spatial attention also remains, because within the object, the cued location still shows greater activity than do uncued locations. Thus, object representations can modulate spatial attention. Can attention to objects also operate independently of spatial attention?

An ingenious fMRI study was done (O’Craven et al., 1999) to address this question. It made use of the knowledge that (a) faces activate the fusiform face area (FFA; see Chapter 6), which is less active in response to images of other objects, such as houses; and (b) a region of parahippocampal cortex is more active in response to images of houses than faces (the so-called parahippocampal place area, or PPA). What was so clever about this study?

FIGURE 7.39 Object representations can modulate spatial attention.
(a) Wrench-like objects were continually presented on the screen and were oriented horizontally (left and middle) or vertically (right). On each trial, a centrally located cue (white arrow) indicated the most likely location of subsequent targets that required a fast response whether at the cued location (frequent) or elsewhere (infrequent). (b) Reaction times to targets were fastest when the cues validly predicted the target location, were slowest to invalid cue trials when the target appeared on a different object, and were intermediate in speed for invalid trials where the target appeared on the same object. (c) Stimulus display for the fMRI experiment, where the upper left location was always cued and where the target appeared on most trials. Uncued locations in the upper right quadrant (for example) could be either on the same object as the cued location (middle) or on a different object (right). The red arrows above each panel indicate the visual field locations corresponding to regions of interest in the visual cortex from which hemodynamic responses were extracted. (d) Hemodynamic responses (percentage signal change) are shown as bar graphs from regions of interest in visual cortical areas V1 to V4. In each area, the largest response is in the cued location, and smaller responses are obtained from uncued locations (the main effect of spatial attention). Importantly, when the uncued location was on the same object as the cued location, the fMRI activation was larger, demonstrating the effect of object attention.

First, the researchers presented superimposed, transparent images of faces and houses so that they occupied the same region of space yet could be seen at the same time (Figure 7.40). Then, they designed the display so that one of the objects moved back and forth while the other was stationary. The motion of the moving stimulus activated cortical motion area MT/V5. Which image moved and which was stationary varied in different blocks. In these different blocks, participants were told to attend selectively to the face, to the house, or to the motion. The activity in the FFA, the PPA, or MT/V5 provided relatively pure measures of the responses to each of these three stimulus dimensions. When participants attended to faces, activity in the FFA increased but activity in the PPA did not; when the participants attended to houses, the opposite pattern of activity was observed. Interestingly, when participants selectively attended to the motion, activity in the MT/MST increased, as did activity in the region (FFA or PPA) corresponding to the object that was moving (face or house, respectively).

The results from the house-face study demonstrate how attention acts on object representations: Attention facilitates processing of all the features of the attended object. For example, face processing was facilitated when the attended moving stimulus was a face, even though the task did not require identification or attention to the face itself. Importantly, these findings show that, when spatial attention is not involved, object representations can be the level of perceptual analysis affected by goal-directed attentional control.

FIGURE 1 Neuronal coherence with attention in visual cortex.
(a) Stimuli consisted of grating stimuli that were in the same V4 receptive field (larger dashed green box) but were in different receptive fields in area V1 (smaller red and blue boxes). (b) Diagram of the left visual cortex of the macaque monkey, showing two regions in V1 (V1a and V1b) that mapped the stimuli shown in (a), as well as how these stimuli were represented in higher order visual area V4. The arrows indicate hypothesized coherences in attention. (c) Neuronal coherence is shown between regions of V1 and V4, depending on which stimulus is attended (see text for more details).

When attention is focused on a stimulus, something happens to the neurons in the visual system that causes the higher visual areas to represent primarily the attended stimulus. Neurons might be modulated by various hypothetical mechanisms, and although we remain uncertain about the precise mechanisms, some interesting models are being tested. One such model suggests that at different stages of visual analysis (e.g., V1 and V4), neurons that code the receptive field location of an attended stimulus show increased synchrony in their activity.

Pascal Fries and his colleagues (Bosman et al., 2012) used cortical surface grids of more than 250 electrodes in the monkey to test this model. They presented monkeys with two drifting gratings separated in visual space, and they trained the monkeys to keep their eyes fixed on a central crosshair but covertly attend one of the drifting gratings at a time to detect when the shape of the gratings changed slightly (Figure 1a). Given the retinotopic organization and small receptive field sizes (~1 degree of visual angle) in V1, stimuli separated by several degrees stimulate different populations of neurons in V1. In higher order visual areas like V4, which have much larger receptive fields (several degrees of visual angle), however, the same stimuli fall within the receptive field of the same V4 neuron (Figure 1a).

The researchers hypothesized that if spatial attention can alter the flow of information from early stages of the visual hierarchy (V1) to later stages (V4) in a spatially specific manner, then this effect might be subserved by selective synchronization of local field potentials (LFPs) between these early and later stages of visual processing (Figure 1b). That is precisely what they observed. They measured the cortical surface LFPs oscillating in the gamma-band frequency (60–80 Hz) and found that coherence increased with spatial attention between the site in V1 coding the attended stimulus location (e.g., location V1a in the figure) and the V4 site coding the stimulus location. So, if the monkey attended location V1a, it showed increased synchronization in gamma-bond LFPs with V4 (Figure 1c, left panel, red). At the same time, however, the coherence remained low between the other V1 location which coded the ignored location (e.g., location V1b in the figure) and V4 (shown in Figure 1c, left panel, blue). Interestingly enough, though, when the animal was cued to switch attention to the other stimulus location (i.e., V1b in the figure), then the V1–V4 coherence went up for that V1 site and V4, and coherence at the first location dropped (shown in Figure 1c, right panel, blue vs. red). These studies suggest that attention alters the effective connectivity between neurons by altering the inter-areal pattern of rhythmic synchronization.

While we most likely have realized from experience that attention affects the processing of perceptual stimuli, we are not conscious of when, where, and to what extent in the chain of processing that attention exerts its effects. The studies presented in this portion of the chapter are beginning to reveal how and where attention affects processing of perceptual stimuli. We now know that visual spatial attention can affect the processing of a stimulus very early in the ascending sensory pathway, and we know where in the cortex this happens. In this section, we also have seen that attention enhances the processing of features of a stimulus. Attention can also be directed at objects. As we have seen, attention mechanisms involve the neuronal machinery specific to processing a particular feature or object. The effects of attention go beyond simple space and feature processing. We have just learned that attention can speed the processing of all the features within an object.

Thus far, we have been considering the influence of attention on sensory processing; we have been looking at the sites of influence of attention. This is only part of the attention story. For the rest of the chapter, we turn to how the focus of attention is controlled.

Control of attention can be both goal directed (topdown) and stimulus directed (bottom-up). Right now, you are using goal-directed attention to focus on this book. But if the fire alarm goes off, your attention will be grabbed by the stimulus, a bottom-up intrusion. Spatial attention is controlled by a mixture of stimulus-driven and goal-directed mechanisms. In goal-directed attention, neuronal projections from executive attentional control systems (with inputs about goals, emotional states, personal experiences, etc.) contact neurons in sensory-specific cortical areas to alter their excitability. As a result, the response in the sensory areas to a stimulus may be enhanced if the stimulus is given high priority, or attenuated if it is irrelevant to the current goal. In contrast, in stimulus-driven attention, the stimulus itself—or some salient features of the stimulus—captures attention, so presumably this process involves circuits from the sensory system interacting with those that orient and engage attention. Selective attention may mediate cortical excitability in the visual cortex through a network that includes at least the posterior parietal cortex, the dorsolateral and superior prefrontal cortex, and the pulvinar nucleus of the thalamus (Figure 7.41). More generally, though, attentional control systems are involved in modulating thoughts and actions, as well as sensory processes.

Studies of patients with either unilateral neglect or Bálint’s syndrome have provided us clues about the control of attention. As noted earlier in this chapter, bilateral lesions to portions of the posterior parietal and occipital cortex result in Bálint’s syndrome, and unilateral lesions of the parietal, temporal, and frontal cortex, especially in the right hemisphere, are implicated in neglect. Neglect may also result from damage to subcortical structures like the superior colliculus and parts of the thalamus. Neurologists, including M. Marcel Mesulam and his colleagues, have described how damage in a variety of these brain areas results in symptoms of neglect (Mesulam, 1981). Mesulam suggested that the disorder of neglect was the result of damage to the brain’s attention network, not the result of damage to a specific brain area (e.g., parietal cortex). What structures constitute the brain’s attentional control network? Does a single network control attention, or are multiple networks involved?

Current models of attentional control suggest that two separate frontoparietal cortical systems are at play in directing different attentional operations during selective attention: a dorsal attention system, primarily concerned with spatial attention, and a ventral attention system, concerned with the nonspatial aspects of attention (Corbetta & Shulman, 2011). It appears that the two control systems interact and cooperate to produce normal behavior. These interactions are disrupted in patients with neglect. These models are based on behavioral studies in healthy persons or in patients with brain lesions, as well as the results of neuroimaging and electrophysiology experiments.

Joseph Hopfinger and his colleagues (2000) and Maurizio Corbetta and his coworkers (2000) both employed event-related fMRI to study attentional control. We reviewed some of the findings from Hopfinger’s study earlier in this chapter, focusing on how spatial attention involves selective processing in visual cortex (the site of attention). Now, we return to this research to see what they learned about the brain regions that control attention.

Finding the Sources of Attentional Control Over Spatial Attention Recall that Hopfinger used a modified spatial cuing paradigm, as shown in Figure 7.15. The participants were presented a cue and were required on some trials to orient attention to one half of the visual field and ignore the other. Then, 8 seconds later, stimuli were presented on both sides of space simultaneously, and the participant was to discriminate target features and make a response. Thus, a goal-directed attentional control network could be identified that was engaged by the appearance of the cue and that was active prior to the appearance of the target stimuli. Such activity can be ascribed to goal-directed attentional control.

What did the researchers find? When the participant attended and responded to the stimulus, a network of dorsal cortical regions showed increased activity. These regions together are called the dorsal frontoparietal attention network. None of the regions in this network were primarily involved in sensory processing of the visual features of the cue, which took place in the visual cortex. We now understand that this dorsal frontoparietal network reflects the sources of attention signals in the goal-directed control of attention. Why did the researchers conclude that these regions are involved in attentional control? First, the identified brain regions were found to be active only when the subjects were instructed (cued) to covertly attend either right or left locations. Second, when the targets appeared after the cue, a different pattern of activity was observed. Third, when participants only passively viewed the presented cues—and didn’t attend to them or act on them—then these frontal-parietal brain regions that were active in the former condition were not activated during passive viewing, even though the visual cortex was engaged in processing the visual features of the passively viewed cues.

The key cortical nodes involved in the frontoparietal network include the frontal eye fields (FEF), located at the junction of the precentral and superior frontal sulcus in each hemisphere, and the supplementary eye fields (SEF) in the frontal cortex; the intraparietal sulcus (IPS); the superior parietal lobule (SPL) and precuneus (PC) in the posterior parietal lobe; and related regions (Figure 7.42). From studies like those from Hopfinger or Corbetta, we know that the dorsal frontoparietal network is active when voluntary attention is engaged. How does this network function to modulate sensory processing?

Linking the Control Network for Spatial Attention to Attentional Changes First, let’s look at the evidence that activity in the frontoparietal attention network is actually linked to attention-related changes in sensory processing. We’ll take another look at Hopfinger’s study (2000). After the cue was presented, but before the target displays appeared, activations were observed in visual cortical regions that would later process the incoming target (Figure 7.43). What caused the visual cortex to be activated even before any stimuli were presented? These activations in visual cortex were spatially specific—they were dependent on the direction of spatial attention. This attentional “priming” of the sensory cortex to a particular location may provide preferential processing to some target inputs (those in that location) over others, a result similar to what has been observed in neurophysiological studies in monkeys (Luck et al., 1997). This priming could be accomplished if neurons in the frontoparietal network send signals either directly or indirectly to the visual cortex, which produce selective changes in visual processing in those visual neurons (e.g., biasing inputs in favor of one location vs. another). Does any data support this biasing effect on the visual cortex?

Frontal Cortex and Attention Control Indirect evidence comes from patients with prefrontal cortical lesions. Neurologist Robert T. Knight and his colleagues (Barceló et al., 2000) found that patients with frontal cortex damage due to stroke had “decreased” visually evoked responses in ERP recordings over visual cortex. This evidence suggests that the frontal cortex (source) has a modulatory influence on the visual cortex (site). More direct evidence comes from intracranial studies in monkeys. As mentioned earlier, a key component of the frontoparietal attention network is the frontal eye fields (FEFs). The FEFs are located bilaterally in the dorsal–lateral–posterior portions of the prefrontal cortex (see Figure 7.42). They coordinate eye movement and gaze shifts, which are important for orienting and attention. Stimulation of FEF neurons produces topographically mapped saccadic eye movements (see Chapter 5). Tirin Moore and his colleagues (Moore & Fallah, 2001) at Stanford University investigated reports suggesting that brain mechanisms for planning eye movements and directing visuospatial attention overlapped. If this is so, then if they altered oculomotor signals within the brain by stimulating them with electrodes, would spatial attention be affected? Using intracortical electrical stimulation and recording techniques in monkeys, they stimulated FEF neurons with very low currents that did not evoke saccadic eye movements. Was there any effect on attention? Yes! While the monkey was performing a spatial attention task (Figure 7.44), the weak stimulations resulted in enhanced performance in the attention task. These effects were spatially specific. That is, attention was enhanced to attended targets only if the targets were at a specific spot. That spot was the location in space where the saccadic eye movements would have been had the stimulation to the FEF been strong enough to generate them: Stimulation of the FEF with currents that do not evoke saccades does bias the selection of targets for eye movements. We now have more evidence that components of the dorsal attention system, in this case the FEF, exerts control over attention.

This finding led the researchers to hypothesize that if FEF microstimulation initiates both saccade preparation and visual selection, then stimulating it also could induce a spatial-attention-like modulation of the visual cortex (Moore & Armstrong, 2003). Again, they placed a stimulating electrode in FEF that could deliver very weak electrical stimulation. This time, they also recorded from V4 neurons whose receptive fields were located in the visual field where stimulation of the FEF would direct a saccade (Figure 7.44a). First they presented a stimulus to the receptive field of the V4 neuron. The stimulus was one of two types: either preferred or non-preferred for that particular neuron. The neuron’s elicited response was always weaker in the case of the non-preferred stimulus. Then stimulation was applied to the FEF site 200–500 ms after the appearance of the visual stimulus. This delay allowed the investigators to examine the effects of FEF stimulation on the activity in V4 that was evoked by the visual stimulus, as opposed to any changes in V4 activity that might have been the direct result of FEF stimulation alone. The FEF stimulation could have had one of three results. It could have amplified the V4 activity, interfered with it, or had no effect on it. What happened? While the monkey was fixating on a central point on the screen, weak stimulation of the FEF-enhanced stimulus evoked V4 activity (i.e., it increased the number of spikes per second) for the preferred over the non-preferred stimulus (Figure 7.44b). If the V4 neuron was not activated by the visual stimulus, then stimulation of the FEF did not affect the activity of the V4 cell. This result mimics the ones observed when monkeys attend and ignore stimuli in V4 (see Figure 7.18). FEF signals appear to participate in goal-directed attentional control over V4 activity.

We have just seen that microstimulation of the FEF in monkeys modulated the neural responses in the posterior visual fields. This is evidence that goal-directed signals from the frontal cortex cause modulations of neural activity. What is the nature of these signals? Are they task specific? For instance, if your task is to identify a face, will goal-directed signals alert only the fusiform face area? Or are signals more broadly transmitted, so that the motion area would also be alerted? Yosuke Morishima and his colleagues (2009) set their sights on answering these questions.

They designed an attention task in which human participants were cued on a trial-by-trial basis to perform a visual discrimination task for either motion direction or face gender. The cue was followed by either a short interval of 150 ms or a long interval of 1,500 ms before the stimulus was presented. The stimulus was a vertical grating that moved to the right or the left, superimposed on an image of a male or female face. In half of the trials, 134 ms after the cue, the FEF was stimulated using transcranial magnetic stimulation.

Recall from Chapter 3 that TMS is a method that uses bursts of focused magnetic fields at the scalp to stimulate neurons in the human brain. Depending on how the magnetic fields are applied, TMS either disrupts or enhances neuronal activity.

Morishima and coworkers used TMS at low enough levels that task performance was unaffected. Thus, TMS did not modify processing in FEF neurons per se, instead it generated a signal in regions of the visual cortex that were functionally interconnected with FEF. Changes in visual cortex activity with TMS were measured by recording ERPs generated by activity of the human motion processing area MT/V5 (MT+) and face processing area (the fusiform face area, FFA). The effect of FEF stimulation on these two brain regions during task performance was evaluated. The results revealed that when participants were cued to discriminate the motion stimulus, the TMS-induced activity in MT/V5 was increased; but when they were cued to discriminate the gender of the face, the same TMS was found to induce increased activity in the face processing region, the FFA (Figure 7.45a). Thus, impulses from the FEF actually coded information about the task that is to be performed, indicating that the dorsal system is involved in generating task-specific, goal-directed attentional control signals.

This study neatly demonstrates that the FEF, a component of the dorsal attention control network, has an influence on visual cortex. This goal-directed influence is task specific, such that the functional connectivity between FEF and specific visual areas is increased as a function of the specific state of attention (i.e., attend face vs. attend motion).

The Parietal Cortex and Control of Attention The posterior parietal lobe is the other major cortical region that is part of the frontoparietal attention system. The parietal cortex occupies a special place in the annals of attention research, owing to the long history of observing that damage to the posterior parietal cortex is related to disorders of attention, such as neglect. We have distinguished between two regions of the posterior parietal lobe (see Figure 7.42). The dorsal areas along the intraparietal sulcus (IPS) and the superior parietal lobule (SPL) belong to the dorsal network, and the ventral areas, which make up part of the temporoparietal junction (TPJ), are a part of the ventral attention network (Corbetta & Shulman, 2002).

The parietal lobe has extensive connections with subcortical areas like the pulvinar and the frontal cortex, as well as other parts of the visual pathways. The parietal lobe contains multiple representations of space. What is the role of the parietal cortex in attention? Numerous physiological studies in monkeys have addressed this question. Attentional shifts are correlated with significant changes in the activity of parietal neurons. Whenever attention is directed to a stimulus, the firing rates of primate parietal neurons increase (Mountcastle, 1976), both when using the stimulus as a target for a saccade or a reaching movement, as well as when covertly discriminating its features (Wurtz et al., 1982). When a monkey is merely waiting for the next trial in a sequence of trials, however, the parietal neurons do not usually show an enhanced response to visual stimuli in their receptive fields (Figure 7.46).

Most studies of attention using single-neuron recording and functional imaging have focused on the intraparietal area, especially the intraparietal sulcus (IPS) and a subregion within the IPS, known in monkeys as the lateral intraparietal (LIP) area (Figure 7.47). This region is involved in the saccadic eye movements mentioned earlier and in visuospatial attention. It is part of the dorsal frontoparietal attention network.

To investigate what role LIP neurons play in visuospatial attention, James Bisley and Mickey Goldberg (2006) collected intracranial recordings of LIP neurons from monkeys as they performed a discrimination task. The monkeys were to detect the properties of a stimulus at a covertly attended location to determine whether to execute a planned saccade to that attended location. While the animal was covertly attending the cued location, occasional distracter stimuli appeared elsewhere. The LIP neuronal activity when there was and was not a distraction was compared. This result was also compared to the monkey’s performance (i.e., its contrast detection threshold) (Figure 7.48).

The best performance was observed when the target feature to be discriminated occurred in the location where LIP neuronal activity was higher. Put another way, if neuronal activity was highest at the attended location, performance was better for targets presented to that attended location. If a distracter had been presented, and neuronal activity had temporarily switched to be higher at another region of the LIP (corresponding to where the distracter was presented), however, then target discrimination was better at that (supposedly unattended) location. For example, Figure 7.48 plots the results from one monkey. Right after the distracter appeared, probe performance was better (panel a, red curve below the blue curve) at the location of the distracter. But around 400 ms (yellow shading) the curves cross. For the remainder of the plot the performance is better at the saccade target location (panel a, blue curve is now below the red curve). This tells us that the distracter briefly captured attention to its location (see Figure 7.29), but then attention returned to the location of the saccade goal. What were the neurons doing at the same time? In Figure 7.48b the red curve plots the neuronal responses evoked by the distracter stimulus and the blue curve shows the earlier saccade goal stimulus at the attended location. When the neuronal response to the distracter is larger than to the saccade goal stimulus, behavioral performance (shown in panel a) is better for the probe at the distracter location. But when the neuronal response to the distracter drops below that for the saccade goal stimulus at around 400 ms, that is when performance crosses back in favor of the attended location for probe discrimination.

Thus, by looking at the pattern of activity over the extent of the LIP, the researchers could actually predict the monkey’s performance. By inference, they also could predict the momentary locus of the animal’s visual attention. Bisley and Goldberg (2006) interpreted this finding as evidence that activity in LIP provides a salience or priority map. A saliency map pools the different individual feature maps (color, orientation, movement, etc.) of a stimulus onto a topographical map, resulting in an overall map that shows how conspicuous a stimulus is from those surrounding it (Koch & Ullman, 1985). This map is used by the oculomotor system as a saccade goal when a saccade is appropriate (i.e., when the stimulus is highly salient). At the same time, the visual system uses this map to determine the locus of attention. Thus, it appears that the LIP, which is an area of the parietal cortex and a component of the dorsal attention system, is concerned with the location and saliency of objects. Let’s now turn our attention to the ventral network.

If a fire alarm goes off while you are reading this chapter, most likely your attention will shift. According to Maurizio Corbetta and his colleagues, this reaction is due to your ventral frontoparietal attention network, which exerts stimulus-driven control. While your dorsal network keeps you focused on the book, the ventral network is standing guard, vigilant for any significant stimuli, at any location in all sensory modalities. This ventral network is strongly lateralized to the right hemisphere. It includes the temporoparietal junction (TPJ) in the posterior parietal cortex, located at the juncture of the posterior temporal lobe and the inferior parietal lobe (Figure 7.49a). The ventral network also includes the ventral frontal cortex (VFC), made up of the inferior and middle frontal gyri.

Corbetta’s group (2002) observed that when a person is selectively attending a region of space, if a relevant stimulus appears somewhere else (from out of the blue, so to speak), a more ventral set of brain areas becomes engaged. In studies that used cues to predict the subsequent target location and cues that did not, the response to stimuli that appeared in unexpected locations activated the TPJ. This region was not engaged during the generation or maintenance of attention, nor with visual searching which engages the dorsal attention network (Figure 7.49b). What did engage the TPJ strongly though was target detection, especially when something occurred in an unexpected location. Interestingly, when this happened, the activity was much greater in the right TPJ. The right TPJ, in fact, responds equally to novel stimuli in both the right and left visual fields (Corbetta et al., 2000).

Similar regions are also engaged by infrequent changes in a stimulus feature, independent of the modality of the change, and by detection of novel stimuli at an expected location. Thus, it seems to be engaged by stimuli that are unexpected, or stimuli that change unexpectedly, what we could call warning stimuli. These are the regions that activate when the birds suddenly stop chirping, or a mouse darts across your room. The TPJ appears to provide an alert that acts like a circuit breaker, interrupting the current attentional focus that is established by the goal-directed dorsal network. Indeed, lesions to the TPJ result in deficits in disengaging spatial attention. Of course, the dorsal and ventral networks are interconnected. Corbetta and colleagues suggest that the dorsal network, specifically the intraparietal sulcus (IPS), provides the TPJ with behaviorally relevant information about stimuli, that is, their salience.

The ventral system is involved with stimulus-driven attention, the detection of salient targets (especially when they appear in unexpected locations), and the reorientation of attention. It is not concerned with spatial attention, per se. Consistent with this lack of concern for space, so far, no topographic maps have been found in the ventral regions.

The regions of the lesioned brain most associated with neglect overlap this ventral attention network (Figure 7.49c). So the dorsal and ventral networks work together. They direct attention to relevant locations and potential targets (frontoparietal system), and they interrupt this attentional state when a novel target appears elsewhere (TPJ and ventrolateral frontal cortex), enabling us to reorient the focus of our attention.

We know something is going on subcortically during stimulus processing, in addition to the activity in the cortex. What is happening there that contributes to attentional control?

Superior Colliculi Changing your focus of attention often involves eye movements, for instance, when you look up from this book to gaze out the window. The superior colliculi, midbrain structures, are involved in this process. They are made up of many layers of neurons that receive inputs from many sources, including the retina, other sensory systems, the basal ganglia, and the cerebral cortex. The superior colliculi project multiple outputs to the thalamus and the motor system that, among other things, control eye movements. Input from the frontal eye fields helps generate intentional saccades, and input from the parietal eye fields aids in triggering reflexive saccades.

In the early 1970s, Robert Wurtz and his colleagues discovered visually responsive neurons in the superior colliculus that were activated based on how monkeys responded to stimuli. Activation required the animal to attend to the location of the stimulus (as is true for cortical neurons) and also to prepare to move its eyes to the target (not necessarily true for cortical neurons). These superior colliculus neurons do not participate in voluntary visual selective attention per se, but are part of an eye movement system and appear to have a role in overt rather than covert aspects of attention. They are sensitive to the saliency of a stimulus (Shipp, 2004), and because of this, they not only detect salient items, but guide eye movements toward them.

Patients with degeneration of the superior colliculus and parts of the basal ganglia, a disease called progressive supranuclear palsy (PSP), have difficulty shifting their attention and are slow to respond to cued targets.

The superior colliculi also appear to be involved with visual search. This was demonstrated by a patient with a rare injury, who, due to bleeding in one hemisphere, damaged only one of the superior colliculi (Sapir et al., 1999). Recall that inhibition of return (IOR) is a bias against reorienting attention to a previously cued location in visual search. This patient had a reduced IOR for inputs to the lesioned colliculus. In the case of reduced IOR, the superior colliculus, in turn, appears to depend on being activated by input from parts of the dorsal network, the frontal eye fields, and the parietal cortex in the hemisphere that is ipsilateral to the site of IOR (Ro et al., 2003).

Pulvinar of the Thalamus One of the many outputs from the superior colliculi goes to a posterior region of the thalamus known as the pulvinar (Figure 7.50). The pulvinar is actually a group of nuclei with connections to many parts of the brain. It has visually responsive neurons that exhibit selectivity for color, motion, and orientation. In addition, it has areas containing retinotopic maps of the visual world and interconnections with frontal, parietal, occipital, and temporal cortical areas. Pulvinar neurons show enhanced activity when a stimulus is the target of a saccadic eye movement or when a stimulus is attended without eye movements to the target. Thus, this structure may be involved in both voluntary and reflexive attention.

To figure out how the pulvinar functions in attention control, Steve Petersen, David Lee Robinson, and their colleagues (Petersen et al., 1987, 1992) chemically deactivated it in monkeys and then observed how their attention changed (Figure 7.51). They injected the drug muscimol, a GABA agonist that inhibits neuronal activity and temporarily deactivates neurons, into the dorsomedial region of the pulvinar. Following the injection, the monkeys had difficulty orienting attention covertly to targets in the contralateral visual field. They also had difficulty filtering distracting information. When competing distracters were present in the visual field, the subjects had difficulty discriminating color or form. Other studies have shown that as the number of distracting stimuli increases, the activity of a normally functioning pulvinar increases (LaBerge, 1990; Buchsbaum et al., 2006). Petersen and colleagues also showed that when bicuculline, a GABA antagonist, was administered, the monkeys readily directed their attention covertly to contralesional targets. Hence, the pulvinar is central to covert spatial attention and filtering of stimuli.

Patients with pulvinar lesions have deficits in attentional orienting. They have a problem engaging attention at a cued location. Compared with normal participants, their reaction times are increased for both validly cued and invalidly cued targets that appear in the contralesional space. This condition is in contrast to patients with cortical lesions of the inferior parietal and temporoparietal junction. Their main deficit was greatly increased reaction times to invalid targets in the contralesional space but not to valid targets (Figure 7.52).

Attention is controlled by what appears to be three interacting networks. The goal-directed dorsal (frontoparietal) attention network is concerned primarily with the control of spatial attention and the saliency of objects. This system enables us to maintain attention on the current goal. It also receives inputs from systems that mediate emotion, memory, and planning. The stimulus-driven-ventral frontoparietal system is essential for disengaging and reorienting our attention. It also detects unexpected, infrequent, or changing stimuli. This system stands guard, ready to shift your attentional focus, if a stimulus is conspicuously different in any way. These two systems interact to allow humans to stay focused on a goal while remaining alert to anything that should warrant our attention. Both are extensively connected to and aided by a subcortical network that contributes to arousal, eye movements, filtering input, and the shifting and orienting of attention.

Summary

If William James and Hermann von Helmholtz were alive today, they would marvel at how much behavioral and physiological data we can provide to answer their questions about attention and awareness. Although in this chapter we did not address all the current information about attention, we did look at key aspects of attentional mechanisms and examined the goal-directed executive systems, and bottomup stimulus-driven mechanisms, that engender orienting and selection within the sensory pathways.

The picture we find is of distributed but highly specific brain systems participating in attentional control. The roles and limits of these systems in attention are becoming more clearly defined as we combine attentional theory, experimental and cognitive psychological findings, and neurophysiological approaches in healthy participants and patients with brain damage. Systems for controlling attention include portions of the parietal lobe, temporal cortex, frontal cortex, and subcortical structures; these comprise the sources of attentional control. The result in visual processing—which has been our example system—is that, in the sensory pathways, we observe modulations in the activity of neurons as they analyze and encode perceptual information as a function of their relevance. These areas affected by attention are the sites of attentional selection.

We no longer wonder whether early or late selection is the mechanism for selective attention, because we now know that attention can operate at multiple stages of processing, including subcortical stages of the sensory pathways. The fascinating fact is that physical stimuli that impinge on our sensory receptors may not be expressed in our awareness, either at the time they occur or later via our recollections. The interaction of stimulus salience and goal-directed attention determine which inputs reach awareness and which do not.

Attentional phenomena are diverse and entail many brain computations and mechanisms. When these are compromised by damage or disease, the results can be devastating for the individual. Cognitive neuroscience is vigorously carving away at the physiological and computational underpinnings of these phenomena, with the dual goals of providing a complete account of the functioning of the healthy brain, and shedding light on how to ameliorate attentional deficits in all their forms.

Key Terms

Bálint’s syndrome (p. 274)

bottleneck (p. 282)

covert attention (p. 280)

dichotic listening (p. 281)

early selection (p. 283)

endogenous cuing (p. 284)

exogenous cuing (p. 296)

extinction (p. 278)

feature integration theory of attention (p. 300)

inhibition of return (IOR) (p. 296)

late selection (p. 283)

limited capacity (p. 282)

neglect (p. 276)

overt attention (p. 280)

pulvinar (p. 320)

reflexive attention (p. 280)

reflexive cuing (p. 296)

selective attention (p. 274)

superior colliculus (p. 320)

thalamic reticular nucleus (TRN) (p. 294)

unilateral spatial neglect (p. 276)

voluntary attention (p. 280)

Thought Questions

Do we perceive everything that strikes the retina? What might be the fate of stimuli that we do not perceive but that nonetheless stimulate our sensory receptors?
Are the same brain mechanisms involved when we focus our intention voluntarily as when our attention is captured by a sensory event, such as a flash of light?
Does attention act on inputs from locations, from object representations, or both? If both, how are the two levels of representation related during selective attention?
Is neglect following brain damage a deficit in perception, attention, or awareness?
Compare and contrast the way attention is reflected in the activity of single neurons in visual cortex versus parietal cortex. Can these differences be mapped onto the distinction between attentional control and attentional selection?