Research

Taking a test often does more than assess knowledge; tests can also provide opportunities for learning. When information is successfully retrieved from memory, its representation in memory is changed such that it becomes more recallable in the future (e.g., R. A. Bjork, 1975); and this improvement is often greater than the benefit resulting from additional study (Roediger & Karpicke, 2006). Interestingly, taking a test can modify memory for information that was not explicitly tested initially (provided that the untested information is related to the tested information in certain ways; Anderson, R. A. Bjork, & E. L. Bjork, 1994; Chan, McDermott, & Roediger, 2006; Hamaker, 1986). Sometimes later recall of this untested information is improved (see, e.g., Hamaker, 1986), but sometimes it is impaired (see, e.g., Anderson et al., 1994), often dependent upon the type of relationship existing between tested and untested information (e.g., Little, Storm, & E. L. Bjork, 2011).

Currently, we are exploring both the benefits and the costs associated with this type of selective testing. This section will focus on situations in which testing some information improves the later recall of untested related information (Note: for information regarding the impairment of related information as a consequence of retrieval, see Retrieval-induced Forgetting).

Recently, we have investigated the potential use of multiple-choice tests as a tool to improve the later recall of untested related information. For example, the recall of initially untested information (e.g., Titan, given the cued-recall question, What is the largest moon of Saturn?) is impaired on a final test as a consequence of trying to recall the answer to a competitive question (e.g., What is the second largest moon of Saturn?, Answer: Rhea) on an earlier cued-recall test; but, the later recall of this competitive information (Titan) can be improved if it is used as a competitive incorrect answer choice on an initial multiple-choice test (e.g., What is the second largest moon of Saturn? A. Titan, B. Rhea, C. Enceladas, D. Mimas). Little and E. L. Bjork (2010) argue that when students do not know the answer to a multiple-choice question, they may try to retrieve information pertaining to why the other answers are incorrect in order to reject them and choose the correct answer. It is this type of processing leads to the spontaneous recall of information pertaining to those incorrect alternatives, thus leading the multiple-choice test to serve as a learning event for both the tested and untested information.

In addition to investigating situations in which tests are used after studying some to-be-learned information, we are investigating situations in which tests are used prior to studying (i.e., pretests). Although pretest performance is poor (because students have not been exposed to the relevant information prior to testing), pretests appear to be beneficial for subsequent learning (e.g., Kornell, Hays, & R. A. Bjork, 2009). We have also investigated the effect of multiple-choice pretests on learning for both pretested and untested related information (Little & E. L. Bjork, 2011). We believe that multiple-choice pretesting is more beneficial than is cued-recall pretesting because the multiple-choice pretest directs attention more broadly during subsequent study–not just to information pertaining to the question, but also to information pertaining to the alternatives.

Memory cues, whether categories, positions in space, scents, or the name of a place, are often linked to many items in memory. For example, the category FRUIT is linked to dozens of exemplars, such as ORANGE, BANANA, MANGO, KIWI, and so on. When forced to select from memory a single item associated to a cue (e.g., FRUIT: OR____), what happens to other items associated to that general, organizing cue? Using the retrieval-practice paradigm, we and other researchers have demonstrated that access to those associates is reduced. Retrieval-induced forgetting, or the impaired access to non-retrieved items that share a cue with retrieved items, occurs only when those associates compete during the retrieval attempt (e.g., access to BANANA is reduced because it interferes with retrieval of ORANGE, but MANGO is unaffected because it is too weak of an exemplar to interfere; Anderson, R.A. Bjork, & E. L. Bjork, 1994, Experiment 3). We argue for retrieval-induced forgetting as an example of goal-directed forgetting because it is thought to be the result of inhibitory processes that help facilitate the retrieval of the target by reducing access to competitors. In this way, retrieval induced forgetting is an adaptive aspect of a functional memory system.

In recent years, we have explored this phenomenon in a variety of ways. For example, we found that items that suffer from retrieval-induced forgetting benefit more from relearning than control items (Storm, E. L. Bjork, & R. A. Bjork, 2008). We have also demonstrated that retrieval success is not a necessary condition for retrieval induced forgetting to occur. That is, when participants are prompted to retrieve with cues that have no possible answer (FRUIT: WO____, rather than the standard, FRUIT: OR_____), access to competing items (BANANA) is impaired, as demonstrated on a final recall test. Furthermore, we are currently exploring the impact of variations of the type of cue support provided for retrieval attempts (FRUIT: OR_____; FISH: ____ORE; WEAPONS: DAGG_____). Our efforts in this domain currently rest on testing various assumptions of theoretical accounts of retrieval induced forgetting.

Sometimes people cannot access information that was well learned earlier (e.g., the address of the house where they grew up). And students find that although they can recall information over and over again the day before a test, they cannot always recall it at the time of examination. Finally, sometimes people cannot recall information at one point in time, but can recall it later. In looking at these situations, it seems that our memories work in strange and unpredictable ways. The function of our memories, however, may be predictable. The New Theory of Disuse (R. A. Bjork & E. L. Bjork, 1992) posits that there are two indices of memory strength: storage strength (SS) and retrieval strength (RS). Storage strength is how well learned something is; retrieval strength is how accessible (or retrievable) something is. To illustrate, imagine four possible situations. If something is well learned (e.g., the address where you have lived for several years), it has both high SS and high RS: You know it well and can retrieve it readily. The address of a friend that you visited for the first time this afternoon, however, may only have high RS (and low SS) because the address, although practiced recently, was not well learned. Thus, although you know the address now, you will be unlikely to be able to recall it in a few days because RS will decrease over time, especially for information with low SS. Sometimes information has high SS (due to it having been well learned), but cannot be retrieved (e.g., the address where you lived as a child). If you were provided with this address again, however, you would have the feeling that that information was somewhere in the recesses of your memory, and in fact, you would be likely to relearn it very quickly. Finally, information can have both low RS and low SS. This information would include things that you heard in class earlier today, but did not learn well and cannot recall now.

The New Theory of Disuse postulates that RS and SS interact in interesting ways. For example, the more SS information has, the bigger the boost in RS it will receive as a consequence of restudy (e.g., you can relearn a childhood address much more quickly than a new address, even if you can’t recall either of them initially). The more RS that information has, the smaller the boost in SS as a consequence of restudy (e.g., simply repeating to-be-learned information over and over again is not very helpful for really learning that information because you are probably practicing rote rehearsal, without forming the deeper connections necessary to improve long-term retention).

The theory has implications for a variety of research projects in the Bjork Learning and Forgetting Lab (e.g., spacing effects, optimizing testing events). Recently, we have investigated situations in which the theory would make unintuitive predictions (based upon previous interpretations of empirical results). For example, Kornell, R. A. Bjork, and Cheung (2009) investigated whether spacing, per se, improves learning or whether the benefit in retention occurs as a consequence of reduced RS at the time of the second learning event. They had participants study passages about various regions of the world (containing information that would be competitive across regions) and then re-study a subset of the studied information (i.e., the earlier learned countries), either immediately afterwards or after a delay. The hypothesis was that the earlier learned information would actually benefit more from a shorter spacing interval than from a longer spacing interval because information at the shorter spacing interval would have lower RS than information at the longer spacing interval (due to retroactive interference). Indeed, their hypothesis was supported, providing evidence for the benefit of re-learning information that is lower in RS–not spacing per se–on later retention. Additionally, Storm, E. L. Bjork, and R. A. Bjork (2008) and Little, Storm, and E. L. Bjork (2011) showed that relearning information that was impaired as a consequence of retrieving other competitive information (i.e., retrieval-induced forgetting) was relearned better after a subsequent restudy event than was information that was also restudied, but not first impaired as a consequence of retrieval-induced forgetting. Finally, John Nestojko and Ben Storm are currently investigating how RS and SS are affected as a consequence of testing versus re-study.

It is common sense that when we want to learn information, we study that information multiple times. The schedules by which we space repetitions can make a huge difference, however, in how well we learn and retain information we study. The spacing effect is the finding that information that is presented repeatedly over spaced intervals is learned much better than information that is repeated without intervals (i.e., massed presentation). This effect is one of the most robust results in all of cognitive psychology and has been shown to be effective over a large range of stimuli and retention intervals from nonsense syllables (Ebbinghaus, 1885) to foreign language learning across many months (Bahrick, Bahrick, Bahrick & Bahrick, 1993).

R. A. Bjork, Kornell and Cheung (2009) argued that the spacing effect, one of the most robust and general findings from the history of experimental psychology, reflects a more fundamental accessibility principle—namely, that reducing the accessibility of information in memory fosters additional learning of that information (see New Theory of Disuse). In this view, increasing the spacing between learning trials enhances learning because it decreases accessibility of the to-be-learned information. Indeed, R. A. Bjork and Allen (1970) found that increasing the difficulty of an intervening distractor task without changing the duration of the spacing interval leads to improved learning, much like a spacing effect. Using a procedure designed to eliminate the normal confounding of spacing and accessibility by inducing absolute recovery—so that accessibility increased, rather than decreased, with delay— R. A. Bjork, Kornell and Cheung (2009) reversed the spacing effect, a result that supports the accessibility principle and runs counter to other explanations of the spacing effect. The accessibility principle may be a kind of law of memory, even if the effect of spacing per se is not.

The study-phase retrieval theory (Thios & D’Agostino, 1975; R. A. Bjork, 1975) of spacing proposed that the benefits of spacing arise from the retrieval of the first presentation upon repeated presentations. Accessibility of the first presentation is diminished at longer intervals and given successful retrieval of the first presentation, longer intervals create more potent learning events than shorter intervals. However, when spacing intervals are too long and repetitions do not retrieve previous presentations, then the study-phase retrieval theory predicts no benefit of spacing. On the other hand, the encoding variability theory of spacing (Estes, 1955) predicts that spacing should be increasingly effective with longer intervals. Appleton-Knapp, R. A. Bjork and Wickens (2005) examined this with an A-B/A-D paradigm and showed that the first presentation (B) is indeed strengthened, providing support for study-phase retrieval.

This theory of spacing would predict that an optimal learning schedule is one with expanding retrieval practice, rather than equally spaced practices. With successive practices, information is better learned and becomes inaccessible more slowly. As the greatest learning occurs when information accessibility is low (but not impossible), increasingly longer lags between retrieval practices should lead to better long-term learning. Indeed, Landauer and R. A. Bjork (1978) showed that expanding retrieval is better than uniform retrieval in two well-controlled experiments.

We are also investigating the impact of spacing on inductive learning. In a first step, Kornell and R. A. Bjork (2008) demonstrated that spacing led to superior inductive learning of artists’ painting styles compared to massing. However, new research from the Bjork Learning and Forgetting Lab shows that it is not necessarily spacing, per se, but rather interleaving that improves inductive learning (Kornell, Birnbaum, Bjork, & Bjork, in preparation; see Inductive Learning).

One robust and longstanding finding is that generating words, rather than simply reading them, makes them more memorable (Slamecka & Graf, 1978). As an example, this effect is often achieved for single words through the use of a letter-stem cue (ex. “fl____” for “flower”) or by unscrambling an anagram (ex. “rolwfe” for “flower”). The effects of generation on memory are being investigated from many different angles in the lab, from its basic role as a memory modifier (see Desirable Difficulties), to people’s awareness of this role and subsequent use of generation as a strategy (see Metacognition), to the extended effects of generation on related material (see Retrieval-Induced Forgetting).

In one study that looked at the effects of generation with typical text materials (DeWinstanley & E. L. Bjork, 2004), it was found that participants who were required to generate certain words (hereafter, target words) in a paragraph had better memory for those words than for target words that were simply read. What’s more surprising, however, is that when these participants subsequently read and were tested on a similar paragraph, again containing target words that were either generated or read, both types of words elicited the same, higher, level of recall. This suggests that the earlier experience of generating words (and specifically, the contrast in memorability between words that were generated and those that were read) made the participants more effective at remembering the paragraphs overall. Subsequent research (Little, Storm & E. L. Bjork, 2011) has indicated that this benefit is due, at least in part, to better memory for the context of the target words.

Spacing is one of the most robust, effective ways of improving learning. However, spacing calls for intervals of time in between repetitions, and this may not be the most efficient use of time. Imagine you have three final exams to study for. If you were to space out study of three whole courses, you might as well start your course review before the quarter even begins! Particularly when one has several different things to learn, an effective strategy is to interleave one’s study: Study a little bit of history, then a little bit of psychology followed by a chapter of statistics and go back again to history. Repeat (best if in a blocked-randomized order).

The benefit of interleaving is found over a diverse set of stimuli ranging from word pairs (Battig, 1979) to motor movements (Shea & Morgan, 1979) to mathematics problems (Rohrer & Taylor, 2007) and word translations (Richland, R. A. Bjork, & Finley, 2004). Interleaving benefits not only memory for what is studied, but also leads to benefits in the transfer of learned skills (e.g. Carson & Wiegand, 1979). The theory is that interleaving requires learners to constantly “reload” motor programs (in the case of motor skills) or retrieve strategies/information (in the case of cognitive skills) and allows learners to extract more general rules that aid transfer.

Benefits of interleaving have also been demonstrated for inductive learning. Although inductive learning occurs in natural settings and everyday life, it can also take place in educational settings. In such cases, it becomes important to direct study in order to optimize learning. One’s intuition may guide him or her to focus on one category at a time before moving on to another (blocking). Kornell and R. A. Bjork (2008) investigated the effects of blocking and interleaving on category induction. Interleaving is to shuffle the exemplars of a given category between exemplars of other similar categories. Interleaving however has the effect of both creating temporal spacing between exemplars of a given category and creating temporal juxtaposition of exemplars of different categories. Kornell, Birnbaum, Bjork, and Bjork (in preparation) investigated these separate effects and found that temporal juxtaposition, which allows for discrimination processes, is more critical to inductive learning than simple temporal spacing is. Additionally, although spacing alone and interleaving alone offer benefits to inductive learning performance (both desirable difficulties, combining the two manipulations does not provide an additive benefit to inductive learning performance (see Interactions between Desirable Difficulties).

Fluency, or the subjective ease of processing information, can provide learners with a useful basis for judging how well information has been understood. Perceptual variations are among the most obvious–and, sometimes, the most misleading—cues to the fluency of information. For example, when you encounter fonts that are difficult to read or words in very small print, you may experience a sense of disfluency—that is, you may have a feeling that the unusual or small typefaces are more difficult to process than more common typefaces.

These types of perceptual cues often cause people to think that disfluent information will be harder to remember than fluent information. Some research, however, indicates that perceptual disfluency can be a desirable difficulty (Diemand-Yauman, Oppenheimer, & Vaughan, 2010). The subjective difficulty of processing disfluent information can actually lead people to engage in deeper processing strategies, which then results in higher recall for those items (Alter, Oppenheimer, Epley, & Eyre, 2007). the Bjork Learning and Forgetting Lab is collaborating with Alan Castel to study how the fluent or disfluent appearance of text (e.g., clear vs. blurred text; upright vs. inverted text) impacts people’s estimations of how well they will remember information as well as their actual memory for that information (Sungkhasettee, Friedman, & Castel, 2011).

It is not enough to simply provide educators with a list of desirable difficulties and claim that our work in optimizing learning has been completed. It may be that certain combinations of desirable difficulties interact to yield super-additive or sub-additive effects, if the processes by which the desirable difficulties work enhance or interfere with each other. Research into these interactions is therefore important on both practical (“What is best for learning?”) and theoretical (“How do these desirable difficulties work to influence memory?”) levels. For instance, research investigating the combination of spacing and variation in advertisements (Appleton-Knapp, R. A. Bjork, & Wickens, 2005) has shown that whilst spacing and variation are each independently ‘desirable’, there is a sub-additive interaction between the two manipulations such that variation is beneficial only when spacing intervals are short. At longer spacing intervals, variation actually hurts memory performance. Furthermore, the results of this study supported the study-phase retrieval theory of spacing over the encoding variability theory of spacing (see Spacing). Current studies in this line of research include examining the interactions between spacing and generation and spacing and variation with educationally relevant materials, such as text passages and glossary-style definitions. Interactions are also being explored with inductive learning, and Kornell, Birnbaum, Bjork & Bjork (in preparation) have found a subadditive effect of spacing and interleaving in the induction of butterfly species (see Interleaving).

Forgetting is often viewed as an uncontrollable, undesirable failure of memory. Yet it is possible to experimentally induce forgetting in an individual that can lead to unexpected benefits. One such paradigm is known as “directed forgetting.” In the typical list-based directed forgetting paradigm (E. L. Bjork & R. A. Bjork, 1996), a participant will study two lists of words, and is notified after each list whether or not it will be tested later on. If a list is tested after the learner was notified that it would not be tested, the learner will show weaker recall for that list, compared to a baseline condition in which all lists are expected to be tested, demonstrating the costs of directed forgetting. Interestingly, it is commonly found that recall of any list that was expected to be tested will be greater than that of the baseline condition, demonstrating the unexpected benefits of directed forgetting.

Another common paradigm for directed forgetting is the item-based method, in which participants are told after each word whether or not it will be tested. A similar pattern of results is observed, in which recall rates for the to-be-forgotten words are depressed, while recall rates for the to-be-remembered words are increased. However, the mechanisms by which item-method directed forgetting occurs are purported to be different than the mechanisms by which list-method directed forgetting operates.

In addition to studying the basic phenomenon of directed forgetting, efforts in the lab are currently underway to further investigate the effects of list-based directed forgetting using different materials and different paradigms. For example, does the pattern of results extend beyond simple word lists to more educationally relevant materials, such as text passages or videos? What happens to the pattern of results when information between the two lists is related? In addition, we are investigating whether directed forgetting applies to other learning paradigms, such as induction learning.

Metamemory can be remarkably accurate in certain contexts, but this is not always the case. For instance, metacognitive monitoring seems to rely on the fluency of an item at encoding (i.e., when it is first learned; for more on this topic, see Perceptual Desirable Difficulties). This is often an accurate basis on which to make judgments, but the Bjork Learning and Forgetting Lab has shown that people still rely on this type of fluency even when it is misleading (e.g., Benjamin, R. A. Bjork, & Schwartz, 1998; Koriat & R. A. Bjork, 2006). Learners also fail to take into account forgetting that occurs over time (Koriat, R. A. Bjork, Sheffer, & Bar, 2004) and learning that occurs with repetition (i.e., the “stability bias”, Kornell & R. A. Bjork, 2009); they instead seem to assume that what they know at the time the judgment is made is an accurate reflection of what they will know at a later time point.

Another important issue in metamemory is people’s awareness of the effectiveness of different study strategies. Zechmeister and Shaughnessy (1980) showed that people tend to think massed repetitions are more effective than spaced repetitions. Massed repetitions lead to greater short-term performance, but impair long-term performance (e.g., Simon & Bjork, 2001); this dissociation could explain why people think massed repetitions are more effective. Kornell and R. A. Bjork (2008) found that in the case of inductive learning, the belief that massed presentation is better than spaced presentation holds true even after people have taken a test and have done better with spacing! At the same time, other recent work (e.g., Benjamin & Bird, 2006; Toppino & Cohen, 2010) has found evidence that people do prefer later re-study to immediate re-study when allowed to choose between the two. Current work in the Bjork Learning and Forgetting Lab is examining further whether and in what contexts people are aware of the benefits of spaced practice when they are allowed to choose how to space repeated study opportunities.