TLDR

• There’s a widespread belief online that it is objectively better to create one’s own cards rather than use pre-made

• The rationale is that self-made cards will ultimately come with better retention

• But not only is research on this mixed, it can actually be poor advice if one does not know how to create good, retainable cards

• The study that is most widely-cited supporting “self-made better” has some important limitations that make it difficult to draw any meaningful conclusions

  • (Strangely I can’t recall or find anywhere that has actually looked into this paper despite it’s frequent citation)

• There are pros and cons to both self-made and pre-made, but ultimately the most important factors are that the cards are sufficient, high-quality and relevant - regardless of how they came to be

• High-quality pre-made cards can actually be uniquely beneficial to ensure one understands the material rather than just the (self-made) card

  • (In an interview, an exam or a conversation with a native it likely won’t be worded the same way one has worded it in a perfectly-worded self-made card)

• Just because a card is self-made does not mean it’s inherently good or better, and vice-versa for pre-made

• Shaeda can (and should) be used for both.

  

  

Intro

It’s often asserted in online discussions that learning via one’s own flashcards is better than using pre-made ones. Whilst it’s not necessarily wrong per se, I suspect that the majority of individuals making this blanket recommendation have simply read it from someone else online and carried on the domino-chain rather than thinking any further about it - which can happen.1

The rationale is actually perfectly fine: self-made cards can come with greater retention (for a mix of reasons). This is good. In fact, it’s very good. After all, strip ‘studying’ or ‘learning’ down and all it ultimately is is gradually increasing the amount of stuff we can (easily) remember. Retention is absolutely front-and-center2.

However, whilst I don’t entirely disagree with the advice nor the rationale, it does require I think some context and nuance which I’ll try to speak to further down. But before that, I want to take look at the widely-cited paper supporting self-made over pre-made.

Research

Open access: User-Generated Digital Flashcards Yield Better Learning Than Pre-Made Flashcards

Abstract & Methods

I’ll try to summarise the paper and their methods as briefly as possible, then provide some screenshots of relevant sections underneath:

• Undergraduate students read 1 page’s worth of material on a random topic for 5 minutes and were told the 10 key terms they had to learn

• After reading for 5 minutes, each student then studied this material using either pre-made or self-made cards for 20 minutes

• After a 5-minute break, each student then read another page of a different random topic and again studied for 20 minutes, but this time using the opposite card method (e.g., pre-made cards topic 1 —> self-made cards topic 2, and vice versa)

• After this second 20-minute block, all studying was finished

• Two days later, the students sat a multiple-choice online test covering both topics

• Averaging across both tests, the self-made group performed better on the quiz (~68% vs. ~60%, on average)

Limitations?

So firstly, as is probably apparent to everyone reading, the setup was not really reflective of any ‘real-world ’ use:

• Reading 1 page of a random topic for 5 minutes

• A single 20 minute session (consisting of optional flipping without grading)

• Then reading a different random topic and studying this for 20 minutes

• 2 days later, sitting a multiple choice quiz

You can compare this to, say, learning a new language, or studying for a degree, where one has to learn several textbooks’ worth of information over 3-10 years and it’s probably not quite the same.

• Card Structure

  

  Fixing the order of cards, preventing ‘hiding’ cards, preventing extra study of certain cards and having optional checking to see the answer defeats a large (in fact, almost the entire) point to using flashcards.

• Ultra-Short Timespan and Content Length

A 5 minute read of a 1 page on a random topic, 20 minutes of interaction with the cards for one session, then a test 2 days later.

There seems to be quite a large misunderstanding or misappreciation of SRS and how it works. It needs time. All learning needs time. If it didn’t, school wouldn’t take us 20 years. Of course it is not feasible to expect researchers to be able to commit to such time-spans due to costs and priorities etc, but it’s important to be aware of just how short this was.

• No Grading

Below is a video of the software used for the flashcards. Not having any form of grading seems very strange. Part of the reason to using flashcards is you are easily able to identify and naturally focus on the content you are struggling to learn, and deprioritise that which you aren’t. Mindlessly flipping-and-repeating for 20 minutes is probably not going to be doing much - if anything. But then again, it was just a single session on a random topic for 20 minutes. Even with grading it would have had only a marginal improvement.

• Low-Quality Pre-Made Cards

This gets to what I mentioned at the start. If the card is low-quality, it doesn’t matter if it’s self-made or pre-made. From the video above (and screenshots below), we can see what the pre-made cards looked like. These are objectively poor quality:

• No way to see the question and answer at the same time

• Far too verbose on the back

• No real context or question is being asked on the front making it harder to apply later on

  

  

Strangely, these appear to have been deemed high-quality by those involved.

• Mixed Findings From Other Studies

The authors provide a table summarising other studies on this, and we can see that it’s actually consistently mixed. Even if we only look at the studies that were of a longer (but still very short) timespan (highlighted in red), we can see it’s still mixed.

I’m going to assume that the limitations present in this study were probably also present in these studies, too. That would make the scattering of results logical: it’s probably just noisy research due to study design.

• No Control (Minor)

I think it would have been very valuable to have had a control topic whereby the students read a page on another random topic but did not use flashcards and instead used only the (sadly) most common forms of studying such as rereading and note-taking etc. I suspect that the performance on this topic’s quiz would have been very poor - in which case the framing of the study may have shifted somewhat.

• Logical Flaws (?)

I could simply be misunderstanding here, but it seems there may have been two potential logic flaws? Below, the authors write that they used a 48-hour delayed test specifically to avoid measuring suboptimal learning techniques such as cramming (whereby the entire deck of cards is studied repeatedly in a single session, typically before an exam)

But as we’ve seen, the setup used was virtually textbook cramming: looping through the same cards continuously for 20 minutes. It seems like they are relying entirely on the two-day delayed test to claim they are measuring durable, long-term learning, even though the study session itself was a one-off 20-minute cramming block (?). Simply delaying the test itself does not render the study session as distributed practice.

Further, it is widely known (and stated) that students benefit more when there is (immediate) correct answer feedback, as is able to occur when cards are flipped over to check for answers - yet as we’ve seen already, this was made optional in the present study?

Pre-Made versus LLM-Made?

I’m going to show a real example of a paid-for (suggesting it should be of higher quality) pre-made card below. It is very low-quality.

Importantly, not necessarily the content on the card, but merely the amount of content and the formatting thereof. This would not at all be very productive for whomever is studying it. It contains 57 words on the back. This is not a cherry-picked example.

But now, if we simply prompt an appropriate LLM with some very basic, effective-flashcard principles, we get the following cards (plural). The plural here is important, as one of the principles to flashcards is keeping them as atomic and as blunt as reasonably possible. That means multiple cards should be used for the same concept.

These are all much higher quality.

Which gets to the main point of this section: LLM-made cards technically are pre-made, but they can actually be customised far more easily than typical pre-made cards and thus made to be very high-quality trivially easily. I suspect that a lot of the reason pre-made has a bad reputation is due to very basic principles not being followed when the cards were created, which ironically then probably fed in to students also creating poor-quality cards. Interestingly, we actually see this being alluded to in wider research whereby teachers are not able to answer student-created flashcards (Dodigovic., 2013).

Language Learning versus Academic Learning

Another key piece of context is what is being learned. If one is studying an academic subject, making your own cards whilst reading a textbook is fairly simple: read, synthesise (whilst trying to avoid overfitting), and type.

However, if you are learning a language, the advice to only “create your own cards” carries a higher risk of actually potentially becoming detrimental.

The reason is essentially one of friction and/or time. When learning a new language, we often cannot just type the Target Language (TL) from our keyboard. We may be able to for some words if it shares the same script, but often times we will not - or at least not easily. Typically one has to constantly stop and copy-paste the native script from elsewhere and then maybe try to find a sentence example, and copy-paste that too. But even if one does manage that, of course an aspect that’s unique to language learning is the necessity of audio. Manually downloading audio files or setting up third-party TTS add-ons for self-made cards is incredibly tedious - in fact many simply put up with not actually hearing the language they’re learning for this very reason.

And even if you do manage to configure audio, you don’t actually have any easy control over the voice or playback speed etc. What ends up happening is that many beginners will simply give up when faced with this initial friction. They’ll either abandon flashcards entirely and jump to a gamified pseudo-learning app, or they’ll try to persevere and download a pre-made deck online only to find out that the cards are not level-appropriate, contain irrelevant cards they wish not to learn yet (for whatever reason), and contain missing audio. Or incredibly loud audio. Or far too quiet audio. Or missing audio. Or audio that you’re not able to playback easily as there’s no ‘play’ button included. The list goes on.

Luckily, Shaeda solves all of these worries.

Summing up

With my own studying over the years, both casual and formal, my self-made cards typically come with about ~10-20% better relative retention per-card. That’s good. The downside, however, is that these self-made decks are around ~5x smaller due to the massive time investment of creating them. For certain cards it was essential, which is fine, but for a lot, it was probably not. I would have been better using Shaeda had it existed.

The trade-off essentially comes down to what I’m very casually and non-precisely terming relative knowledge and absolute knowledge (better and more precise terms exist3):

• Relative Knowledge (RK): how easily one can recall each piece of content learned, regardless of volume.

• Absolute Knowledge (AK): the total volume of content that has been learned.

When we first begin learning, say, a new language or a new subject, both domains are low - potentially even zero. As one begins to learn, RK increases faster (in percentage terms) than AK. After a few days of learning, say, Chinese, I may have studied 10 words and successfully been able to recall 8 when tested by a teacher (or on Shaeda’s diagnostic test). That’s an 80% RK. Unfortunately for me it takes around ~5,000 words to become fluent, so my AK of 8 is just 0.16% - not quite these guys just yet. Consider the flip-side of, say, a retired professor: an incredibly high AK, but their RK has likely dropped.

If one is aiming to learn several textbooks’ worth of content over many years, it would probably be somewhat strange to celebrate, say, a 90% recall of book 1, chapter 1 after a year of study over, say, an 80% recall of 3 entire books. Here, RK has been maximised at the expense of AK. There’s many instances where this will be fine, correct even, but given the initial goal was to learn several textbooks, it is probably detrimental.

The concern and/or risk of pre-made is that the cards may be low quality - and one would not know until they view them for the first time. But this is not unique to pre-made. It can apply just as much to self-made, which we’ve already seen (Dogovic., 2013). After all, every pre-made deck was self-made to someone, somewhere.

Ultimately, all-else-equal, the act of creating one’s own cards will likely be contributing to the learning effect - albeit very slightly. That part is true. But whether that means that one should use and only use their own self-made cards is another question entirely.

Takeaways:

• Quality matters much more than authorship

• The frequently-cited paper supporting “self-made always” has many limitations that are seemingly not widely appreciated

• Language learning or Academic learning is important context

• Leveraging LLMs correctly can be incredibly (and uniquely) beneficial

• Use a lot of both, just make sure they’re good4

PS: A lot of users who have signed up for Shaeda have all these posts go to their spam - this isn’t ideal. If you found this post insightful, please consider liking and sharing. Substack say that this will help emails be directed to inboxes instead. Thank you!

Share

Subscribe now

• 90 final-year undergraduate medical students were randomly split into two groups over 6 months: one group used Flashcards, and the other carried on with ‘traditional study’

• Both groups began the study with roughly the same knowledge levels (potentially higher in the Traditional Study group)

• After 6 months, the Flashcard students test performance improved significantly, while the traditional group’s did not improve virtually at all.

• Over 90% of students in the Flashcard group reported better retention, engagement, and confidence, with ~65% saying that they felt so “very strongly”.

TLDR Study 2:

• This was an observational study (researchers watched from afar) looking at 36 medical students who were already using flashcards

• Mature cards (cards that have a long gap [interval] due to repeated success in consistently recalling) were the strongest predictor of exam performance

• High flashcard users (users that had amassed ≥9,390 mature cards) scored 71.5%

• Low users (<9,390 mature cards) scored 60.0%

• ~10 percentage point difference between high and low users

• Overall, students averaged around 4 hours per day studying

A couple of months ago, we covered some research on what effective studying looks like (according to a lot of research) and whether or not flashcards are effective. You can take a look here.

Since then, more research has came out that has tried to explore this further. This will be a quick blog post that goes into these studies.

Both papers are open-access, and are linked and cited below.

Study 1:

Notes on Method

Design and Cohort

Rather than employing true randomisation, this study utilised a quasi-experimental design where participants were allocated to groups based on their clinical rotation schedules. While this non-randomized approach can introduce certain methodological limitations, baseline assessments demonstrated that the groups possessed equivalent prior knowledge.

Duration and Spacing

Notably, the 6-month tracking period is a very welcome and uncommonly-long window for educational research evaluating spaced repetition systems (SRS). This extended duration is very relevant, as brief study windows often underestimate the compounding benefits of the spacing effect. Consequently, shorter-term literature is almost certainly systematically underreporting the true efficacy of paired retrieval practice and spaced recall.

Demographics and Algorithms

The cohort consisted of final-year medical students characterised by advanced academic training and high baseline knowledge. However, the study relied on fixed review intervals, which are generally suboptimal compared to adaptive systems. Implementing a more modern spacing algorithm would likely yield superior retention outcomes while optimising time efficiency. Minimising the daily review burden addresses a primary barrier to flashcard adoption, making the potential efficiency gains of adaptive algorithms a highly consequential area for optimisation.

Notes on Results

Notably, the control group exhibited a marginally higher baseline knowledge score compared to the intervention group. While this initial difference did not achieve statistical significance, it closely approached the conventional threshold of p <0.05 (p=0.0573). This asymmetry is relevant to the evaluation of the intervention’s efficacy - because the flashcard cohort began at a relative disadvantage, their subsequent substantial performance gains underscore the effectiveness of flashcards (they started with a slightly lower knowledge, and ended with a significantly higher)

Although the absolute mean difference between groups was modest (approximately 5 points), the observation was highly statistically significant (p<0.0001), which indicates a negligible chance that the outcome was due to random variation. While standard literature correctly cautions against over-interpreting p-values in isolation or equating them blindly with clinical significance, a result of this degree gives good evidence that the improvement cannot be attributed to mere chance.

Now let’s get into the bulk of the paper. Feel free to scroll past all this if you’ve seen enough and would like to see about study two.

Barriers to Flashcard Adherence

A main barrier to the adoption of retrieval practice is the perception of time constraints. Learners frequently operationalise this by prioritising passive study strategies (such as rereading, passive note-taking, and highlighting) while treating flashcard-based self-testing as a peripheral or secondary activity. This preference is pedagogically flawed. Research consistently demonstrates that passive re-exposure yields inferior long-term retention compared to active retrieval, despite the higher subjective cognitive effort required by the latter. The reason passive study methods are so popular and common is that they feel effective, lulling one into a false sense of familiarity and progress. This is the very basis that 99% of online study apps are based on.

Similarly, adherence to spaced retrieval regimens can often be erratic. Maintaining a consistent daily review schedule is modulated by several variables, both structural and intrinsinc:

• Effective flashcards rely on the minimum effective context principle, requiring information to be atomic and direct. In contrast, overloading items with complex, multi-sentence textual blocks violates this principle, increasing friction and decreasing processing efficiency. For more on this, see here.

• Exceeding an optimal threshold of novel items per day induces cognitive overload. This (excessive) volume slows the rate of memory consolidation and rapidly degrades both the efficiency of the study session and adherence.

• Certain academic disciplines (particularly algorithmic or problem-solving domains such as computer programming and applied mathematics) are often thought as being less inherently suited to conventional declarative flashcard formats. However, spaced retrieval remains necessary in these fields to maintain the baseline factual framework required for higher-order problem-solving.

• Learner sentiment and topic interest heavily moderate study efficacy. When intrinsic motivation is low, passive methods exacerbate inefficiencies. In low-interest domains, highly structured, time-efficient retrieval practice is crucial to maximize the utility of the limited study time a learner is willing to invest.

Some Notes:

• The flashcards were custom made by senior staff, professors, or postgraduate students. This is not to say that self-made or pre-made (online) are going to be worse per se. It depends entirely on the quality of the card. Even teachers can make low-quality cards.

• This is why all subjects and languages on Shaeda (will) have undergone validation

  • (Although per the previous blog, a custom/DIY flashcard feature will also be added into Shaeda as well.)

  • For info on what ‘validation’ looks like (or rather, what it results in), see below:

    

• As I’ve already touched on before, though, there is always a trade-off at play [with everything]. That is, yes, custom-made flashcards from 1:1 tutoring with a tutor can result in great cards, with perfect audio (for language learning), or perfect examples/explanations (for academic learning). Whether this translates, however, into better knowledge and retention is another question.

  • It will also, naturally, cost significantly more - along with being a much slower process of obtaining new cards. This is the trade-off. Unfortunately, there’s no free lunch.

Study 2:

Quick Thoughts on Table 5

When we look at study data, different factors are often closely linked. For example, students who study for more hours often have more active flashcards. To prevent these overlapping factors from muddying the results, the authors used a statistical method called multiple linear regression. This essentially allowed them to ask: “If two students studied for the exact same number of hours, but one successfully remembered more cards, who did better?”

This statistical model accounted for nearly 43% of the differences in student exam scores (R2 = 0.427). When comparing all three habits side-by-side, Mature Cards (cards kept in rotation long enough to be successfully recalled multiple times) stood out as the only factor that independently predicted higher scores.

The other two factors measured had almost no independent impact:

• Unsuspended Cards (Active Cards): The raw number of active cards in a student’s deck had zero real effect on performance. The data showed this result was statistically meaningless and likely just random background noise.

• Total Hours Studied: Surprisingly, raw study time had virtually no independent impact on exam scores once we controlled for the other habits.

Overall it shows that simply owning a massive flashcard deck or putting in long hours of raw study time doesn’t automatically guarantee success. Instead, true academic performance is driven by how many cards a student actually masters and remembers over time. When someone claims that flashcards “don’t work,” the data suggests we need to look closely at how they are managing their study habits.

Other Potential Limitations

• No baseline knowledge assessment

  • Students may have entered with vastly different foundational knowledge, making it impossible to isolate Anki's effect from prior learning

• No control for baseline study skills:

  • Students with better existing study habits might naturally gravitate toward systematic tools like Anki

That’s all.

A small reminder that likes and shares are appreciated as it improves the ‘health’ of the Substack, meaning more people who subscribed will actually receive the emails in their inbox as opposed to their spam.

Thanks for reading.

Share

Subscribe now

Citations:

Vagha, K., Choudhari, S., Taksande, A., Tembhurne, J., Vagha, J., & Vagha, S. (2025). Implementation of a spaced-repetition approach to enhance undergraduate learning and engagement in paediatrics. Frontiers in medicine, 12, 1601614. https://doi.org/10.3389/fmed.2025.1601614

Winter, V., Ames, E., Jacobs, M., Georges, C., & Simanton, E. (2025). Exploring the Impact of Spaced Repetition Through Anki Usage on Preclinical Exam Performance. Journal of Medical Education and Curricular Development, 12. https://doi.org/10.1177/23821205251369705 (Original work published 2025)

• A new paper just published may give more evidence that flashcards are both (very) effective and under-utilised

• A 2021 meta-analysis appears to find that only 2 methods of studying are highly effective (all others are of low or moderate effectiveness)

• Majority of students seem to consistently use the lowest-effectiveness methods (which will likely only get worse with the rise of AI)

• Some new shaeda updates

• A very small request at the end

• Research seems to indicate that a large percentage of teachers during their training were not made aware of the research on effective studying and learning

Are Flashcards Effective?

A new study by Ingebrigtsen and colleagues testing the effectiveness of flashcards on student learning was just published the other day (Ingebrigtsen et al., 2025). The discussion segment, which begins at the bottom of page 10, is particularly insightful without needing to read the entire paper necessarily. I will paste the Abstract below.

The below is a very quick summary:

• Students were randomised into whether they were placed into the flashcard group or not

• The exam was a first-year university exam covering Anatomy, Physiology and Biochemistry with a historically high failure rate of ~20-25%.

• Flashcard users scored on average +7.6 points greater on the final exam (out of 100)

• Flashcard users were nearly 3× more likely to pass

• Flashcard users were over 2× more likely to get an A

• The relationship between flashcard use and greater exam performance was maintained after adjusting for potential confounders (age, effort, study duration etc)

I will paste some screenshots of the paper that are of most relevance.

(Related, see Self-Made Vs. Pre-Made Flashcards)

Do Related Studies Find Similar Results?

The study above builds on previous papers which also found positive effects of spaced practice/flashcards/testing:

This paper concluded “A total of 87% of flashcard users found the flashcards to be helpful, and 83% of flashcard users would recommend the flashcards to someone else. Flashcard usage was not associated with final exam scores.” (Sun et al., 2021)

Some notes here:

• They were restricted from actually publishing the exam results

• Flashcards were only used for a short period of time, thus the full effects were not likely to have fully materialised yet

The authors of this paper write the following in an ‘Essentials’ segment (Deng, Gluckstein & Larsen., 2015):

Remember, at a fundamental level all flashcards are doing is easily implementing Spaced Practice and Self-Recall (i.e., retrieving from memory with minimal (0) assistance)

For more on these 2 methods [Spacing x Recall], I will paste some quotes from a meta-analysis performed in 2021 below.

2021 Meta-analysis on Effective Study Techniques

The below is directly quoted from this Meta-analysis where the authors are referring to a previous meta-analysis done in 2013.

This article outlines a meta-analysis of the 10 learning techniques identified in Dunlosky et al. (2013a), and is based on 242 studies, 1,619 effects, 169,179 unique participants, with an overall mean of 0.56.

The most effective techniques are Distributed Practice and Practice Testing and the least effective…are Underlining and Summarization

The authors of a previous meta-analysis categorised 10 techniques into three groups based on whether they considered them having high, medium or low support for their effectiveness in enhancing learning.

• High support ✅

  • Practice Testing

  • Distributed Practice

• Moderate support

  • Elaborative Interrogation (generating an explanation of a fact or concept)

  • Self-Explanation (where the student explains how new information is related to already-known information)

  • Interleaved Practice (implementing a schedule of practice mixing different kinds of problems within a single study session).

• Low support

  • Summarization (writing summaries of to-be-learned texts)

  • Highlighting/Underlining (marking potentially important portions of to-be-learned materials whilst reading)

  • Keyword Mnemonic (generating keywords and mental imagery to associate verbal materials)

  • Imagery use (attempting to form mental images of text materials while reading or listening)

  • Re-Reading (restudying text material again after an initial reading)

In an accompanying article, Dunlosky et al. (2013b) claimed that some of these low support techniques (that students use a lot) have “failed to help students of all sorts” (p. 20), the benefits can be short lived, they may not be widely applicable, the benefits are relatively limited, and they do not provide “bang for the buck” (p. 21).

Practice Testing is one of the two techniques with the highest utility. Practice Testing instead involves any activity where the student practices retrieval of to-be-learned information, reproduces that information in some form, and evaluates the correctness of that reproduction against an accepted ‘correct’ answer. Any discrepancy between the produced and “correct” information then forms a type of feedback that the learner uses to modify their understanding. Practice tests can include a range of activities that students can conduct on their own, such as completing questions from textbooks or previous exams, or even self-generated flashcards. According to Dunlosky et al. (2013a), such testing helps increase the likelihood that target information can be retrieved from long-term memory and it helps students mentally organize information that supports better retention and test performance. This effect is strong regardless of test form (multiple choice or essay), even when the format of the practice test does not match the format of the criterion test, and it is effective for all ages of student. Practice Testing works well even when it is massed, but is even more effective when it is spaced over time. It does not place high demand on time, is easy to learn to do (but some basic instruction on how to most effectively use practice tests helps), is so much better than unguided restudy, and so much more effective when there is feedback about the practice test outputs (which also enhances confidence in performance).

Many studies have shown that practice spread out over time (spaced) is much more effective than practice over a short time period (massed)–this is what is meant by Distributed Practice. Most students need three to four opportunities to learn something (Nuthall, 2007) but these learning opportunities are more effective if they are distributed over time, rather than delivered in one massed session: that is, spaced practice, not skill and drill, spread out not crammed, and longer inter-study intervals are more effective than shorter. There have been four meta-analyses of Spaced vs. Massed practices involving about 300 studies, with an average effect of 0.60 (Donovan and Radosevich, 1999; Cepeda et al., 2006; Janiszewski et al., 2003; Lee and Genovese 1988). Cepeda et al. (2008) showed that for almost all retention intervals, memory performance increases sharply with the length of the spacing interval.

Further, Spaced Practice is more effective for deeper than surface processing, and for all ages. Rowland (2014) completed a meta-analysis on 61 studies investigating the effect of testing vs. restudy on retention. He found a high effect size (d = 0.50) supporting the testing over restudy, and the effects were greater for recall than for recognition tasks. The educational message is to review previously covered material in subsequent units of work, time tests regularly and not all at the end (which encourages cramming and massed practice), and given that students tend to rate learning higher after massed, educate them as to the benefits of spaced practice and show them those benefits.

A follow-up and more teacher accessible article by Dunlosky et al. (2013b) asks why students do not learn about the best techniques for learning. Perhaps, the authors suggest, it is because curricula are developed to highlight content rather than how to effectively acquire it; and it may be because many recent textbooks used in teacher education courses fail to adequately cover the most effective techniques or how to teach students to use them. They noted that employing the best techniques will only be effective if students are motivated to use them correctly but teaching students to guide their learning of content using effective techniques will allow them to successfully learn throughout their lifetime.

The authors of this paper then essentially tested again this 2013 meta-analysis to calculate an individual effect size for each of the 10 learning techniques.

Method:

Research syntheses aim to summarise past research by estimating effect-sizes from multiple, separate studies that address, in this case, 10 major learning techniques. The data is based on the 399 studies referenced in Dunlosky et al. (2013a). We removed all non-empirical studies, and any studies that did not report sufficient data for the calculation of a Cohen’s d. This resulted in 242 studies being included in the meta-analysis, many of which contained data for multiple effect sizes, resulting in 1,620 cases for which a Cohen’s d was calculated (see Figure 1).

Results:

Table Overview

In the first column we see the study technique.

In the second we see what the 2013 meta-analysis classified each as in terms of effectiveness.

The third, # Cases, represents the total number of effect sizes. Since a single study can report multiple results (e.g., for different groups or at different times), this number can be higher than the number of studies. We can see that there’s much more data for the top 2 techniques.

The fourth column, Unique N, is the total number of unique participants (subjects) across all studies for that technique. N is a standard abbreviation for number. Using a "Unique N" ensures individuals are not counted multiple times, giving us the true sample size. Generally, a larger N is better as it increases the statistical power and precision of the results.

The fifth, d (short for Cohen’s d), is arguably of most relevance and hence why I’ve highlighted it. This is the actual calculated effect size. It attempts to tell us how much of an effect did the study method have on knowledge retention. Below is loosely how to interpret:

• d of <0: Negative effect.

• d of 0: No effect.

• d of >0: Positive effect;

  • ~0-0.2 = Small positive effect

  • ~0.2-0.5 = Moderately positive effect

  • ~0.8 = ~High positive effect

The sixth column, SEM (Standard Error of the Mean), attempts to quantify the precision of the calculated mean effect size. It represents the estimated SD of the sample means if you were to re-run the study multiple times. It essentially serves as a measure of confidence in the result: the lower the SEM, the more precise the estimate and the more likely it is that our sample mean accurately reflects the true population mean.

The seventh column, Q, measures how much the results of the grouped studies varied from each other. I.e., how different their findings were. We can see that there was a lot of variation. For more on this would be beyond the scope of this email, but the authors address it in their paper with further analyses.

And lastly we have I² (I-squared) which is telling us what percentage of total variation in the results is due to the differences between studies rather than random chance. Again, more on this would be beyond the scope of this email but can be found by reading through the paper.

The purpose of the current study was twofold: to provide empirical estimates of the effectiveness of the 10 learning techniques, and second, to empirically evaluate a range of their potential moderators.

The technique with the lowest overall effect was Summarization. Dunlosky et al. (2013a) note that it is difficult to draw general conclusions about its efficacy, it is likely a family of techniques, and should not be confused with mere copying. They noted that it is easy to learn and use, training typically helps improve the effect (but such training may need to be extensive), but suggest other techniques might better serve the students.

In their other article (Dunlosky et al., 2013b), the authors classified Summarization as among the “less useful techniques” that “have not fared so well when considered with an eye toward effectiveness” (p. 19).

What is now most startling is that it appears quite consistently that the least effective techniques are actually the most used by students when asked.

Remember that the authors of the first study on flashcard effectiveness also cited other studies and wrote similar:

Awareness in Schools

Further Reading

For more reading on this for those interested, I can recommend the following books which are all both recent and based on research.

How Learning Happens

How We Learn

That is everything. Hopefully this may be helpful and reassuring to many who are confused about flashcards and/or wondering what the research finds regarding their effectiveness.

If you know someone who may be interested in the research shared in this post please do consider sharing this with them to help their studying, too.

Share

Subscribe now

Bibliography

Ingebrigtsen, M., Odden Miland, Å., Basteen, J., & Grom Sæle, R. (2025). Effective, scalable, and low cost: The use of teacher-made digital flashcards improves student learning. Applied Cognitive Psychology, 39(1), e4254. https://doi.org/10.1002/acp.4254

Donoghue, G. M., & Hattie, J. A. C. (2021). A meta-analysis of ten learning techniques. Frontiers in Education, 6, Article 581216. https://doi.org/10.3389/feduc.2021.581216

Urrizola, A., Santiago, R., & Arbea, L. (2023). Learning techniques that medical students use for long-term retention: A cross-sectional analysis. Medical teacher, 45(4), 412–418. https://doi.org/10.1080/0142159X.2022.2137016

Larsen DP, Butler AC, Roediger HL 3rd. Repeated testing improves long-term retention relative to repeated study: a randomised controlled trial. Med Educ. 2009 Dec;43(12):1174-81. doi: 10.1111/j.1365-2923.2009.03518.x. PMID: 19930508.

Kerfoot BP, DeWolf WC, Masser BA, Church PA, Federman DD. Spaced education improves the retention of clinical knowledge by medical students: a randomised controlled trial. Med Educ. 2007 Jan;41(1):23-31. doi: 10.1111/j.1365-2929.2006.02644.x. PMID: 17209889.

Sun M, Tsai S, Engle DL, Holmer S. Spaced Repetition Flashcards for Teaching Medical Students Psychiatry. Med Sci Educ. 2021 Apr 6;31(3):1125-1131. doi: 10.1007/s40670-021-01286-y. PMID: 34457956; PMCID: PMC8368120.

Deng F, Gluckstein JA, Larsen DP. Student-directed retrieval practice is a predictor of medical licensing examination performance. Perspect Med Educ. 2015 Dec;4(6):308-313. doi: 10.1007/s40037-015-0220-x. Erratum in: Perspect Med Educ. 2016 Dec;5(6):364. doi: 10.1007/s40037-016-0312-2. PMID: 26498443; PMCID: PMC4673073.

Piza, F., Kesselheim, J. C., Perzhinsky, J., Drowos, J., Gillis, R., Moscovici, K., Danciu, T. E., Kosowska, A., & Gooding, H. (2019). Awareness and usage of evidence-based learning strategies among health professions students and faculty. Medical teacher, 41(12), 1411–1418. https://doi.org/10.1080/0142159X.2019.1645950

Demekash, A. M., Degefu, H. W., & Woldeab, T. A. (2024). Secondary school EFL teachers' awareness for formative assessment for effective learning. Heliyon, 10(19), e37793. https://doi.org/10.1016/j.heliyon.2024.e37793