Hacker Newsnew | past | comments | ask | show | jobs | submitlogin
Spaced Repetition (gwern.net)
252 points by jger15 on May 4, 2019 | hide | past | favorite | 68 comments


I had considered using spaced repetition software for a while after a long time of being frustrated with my poor memory. I finally started using Anki last September after reading Augmenting Long-term Memory[1].

I started off with just putting in a new vocabulary word every day and have been gradually ramping up the amount of info that I put into it. Now, I will add anything from programming details (e.g. how do you get the number of characters in a Rust string?) to key facts from the books I read (e.g. in The English Patient, who does the patient have an affair with?). My review sessions only take 5-10 minutes a day, and my only regret is not starting this habit earlier (especially in school). It's incredibly gratifying to review something, know the answer, and be very aware that without Anki, I would have forgotten it a long time ago.

[1]: http://augmentingcognition.com/ltm.html


Do you use multiple decks? I found myself creating several, and then finding it hard to follow up on many of them. It’s a pity though, because some of the cards in them were interesting, but the whole deck wasn’t. Maybe I should have a “misc” one.


I don't. Everything goes into a single deck so that my review sessions interleave[1] topics.

But if you have cards that you don't find interesting or useful anymore, maybe it'd be better to just delete them.

[1]: https://www.scientificamerican.com/article/the-interleaving-...


Doesn't that mean you have to manually put the subject into each card?

E.g. I am currently using Anki to learn more keyboard shortcuts for Bash, Tmux, and vim, and the deck tells me which program the card is talking about. I already have to put context for vim visual mode keys in those cards.

Edit: Ah, just saw your other comment about doing something like it with tags.


I still manually add context to each card. I want each card to be self-sufficient, even without any tag data.


Considering how many people are learning Rust, I think that Anki Rust cards could be a huge hit. What do you think?


That could be helpful for some, but personally, I don't use any premade decks because the act of making cards is part of the learning process.


very interesting, i would also like to know how you organise your decks/tags


I use a single deck, which seems to be a commonly accepted best practice for Anki because mixing topics is good for learning.

I do tag my cards. For example, if a card is "In HTML, what header should you use to make a website responsive?", I would tag it with "programming," "webdev," and "html." The organization is there, but I haven't actually found a use case for the tags yet.


My plan is to use tags for deleting cards. For example, when I take another job/role then lots of cards should become obsolete.


I've been doing the "30 hour method" as described here: https://azeria-labs.com/the-importance-of-deep-work-the-30-h...

It's sort of like the Pomodoro technique, except with 30-hour chunks instead of 30-minute chunks. (Obviously they serve different purposes.)

After studying a topic, while it is still fresh in my mind, I will make a few entries into my spaced repetition system (org-mode) and then I only review the entries between each of the 30-hour blocks. This means I'm only reviewing entries maybe twice a month at most. Yet, the spaced repetition still works well, and surprisingly, it doesn't take long before I'll only have one or two entries to review during my rare study sessions.

My point being, even if you study your spaced repetition entries only once a month, it will still work.

It also helps me not become too obsessed with creating cards for every little thing. I study a topic for a few weeks, and only at the end do I even think about spaced repetition and making cards. I review what I've learned (I do take notes sometimes) and make only a few cards for the most important topics. Ultimately I'm only thinking about or interacting with spaced repetition systems one or two days a month. Again, they still work even when used so rarely.


+1 for using org-mode and org-drill.

Also means that you can export it all to a well structured PDF and use on any device that can read PDFs.

Downside is one cannot do drills unless at the PC.


... another downside is that I don't know how to include audio (useful to learn to pronounce a foreign word).


What are some of the things are you studying? Do you do 2 sessions a week? For 7 sessions, so that's 3.5 weeks, and then an extra session on the review/spaced repetition?


If anyone's studying Japanese, I made two iOS apps for spaced repetition systems specializing in the unique needs of learning Japanese.

The first is a flashcard application called Manabi. It uses a similar algorithm as Anki with a priority on UX.

https://manabi.io

The other is Manabi Reader, which complements the flashcard app. It curates feeds of interesting short-form content, has one-tap dictionary lookups, optional furigana injection and JLPT level tagging, and it lets you create flashcards from the words you're learning as you read.

https://reader.manabi.io

Glad to hear any feedback. I'm currently working on word tracking functionality for Manabi Reader, to track your reading progress and let you see which and how many words in an article you should know already vs. words new to you.


This looks really cool. Do you have plans for an android version? I wouldn't mind paying for this.


Sorry, no, they’re completely native apps so I would have to rewrite from scratch on Android. Would rather focus my time on improving these. Cheers.


That's too bad :(. Although, I admire your attitude of focusing on improving the iOS apps rather than branching out.


I mentioned it before, but https://ncase.me/remember/ is a verily accessible read on the subject of spaced repetition.


I have often tried finding the actual research articles that describe the increasing half-life at the "nearly forgetting" point.

Does anyone know where I can find the equations that model this qualitatively? (of course there's no expectation that it will list actual quantitative coefficients for my brain)


Scroll to the bottom of the SuperMemo homepage (https://www.supermemo.com/en) and look at the list of articles. Some of them are rather informal, others are more academic, although I'm unsure if they were peer reviewed and published or not.


thanks for roughly pointing me in the right direction, but at http://super-memory.com/help/fi.htm it states:

>It is difficult to determine exactly what forgetting index brings the highest acquisition rate. Simulation experiments have consistently pointed to the value of 25-30%. You can even plot speed-vs.-forgetting graph using your own actual learning material in SuperMemo 98 or later using Tools : Statistics : Simulation. You will probably also arrive at similar results

How can simulation provide us with actual real world data?

How do we know that the forgetting index sweet spot is independent of the number or even history of recall attempts? If I write R to denote remembered and F as forgotten, then I can imagine RRF, RFR, FRR (variable ordering of failures for a constant recall rate), and RRF, RFF, FFF (variable retention rate for different factoids), and R, RR, RRR (number of recall attempts) to influence the sweet spot level.

Does Anki support plotting plotting such a user's graph? If it does I might consider rewriting the code such that it can generate multiple plots (to find sweet spot at N-th recall attempt, N-th recall attempt with M succesful out of N-1 previous recall attempts, ...)

in the above "simulation experiments" links to http://super-memory.com/articles/theory.htm#maximum%20speed

which doesn't describe any simulation, but just shows a plot, presumably generated by supermemo statistics for an individual?

If someone has a large homogenous set of Anki cards, I'd like to experiment myself. Perhaps vocabulary of a foreign language? If I'm going to use a vocabulary dataset, I'd need a dataset where the responses are sense disambiguated though (i.e. the sense in parentheses), otherwise I am cramming wrong associations.


This page is great! Just did it.


If anyone's interested, I've compiled a lot of strategies and tips around spaced repetition and Anki that I've learned over five years and ~11k cards.

https://senrigan.io/blog/everything-i-know-strategies-tips-a...


Neat! I have a similar page I put up at 10k cards. We should be friends

http://rs.io/anki-tips/


YOU'RE ALIVE! I spent a few hours two years ago wondering what had happened to you (your blog and twitter went quiet). I had presumed the worst.

Thought rs.io is/was really well done, great curated stuff. Added a reference link to your pages.

Hit me up at jeffshek@gmail.com anytime! I sheepishly already follow you on Twitter lol.


Thanks! There are some good insights in both of your articles. In particular I realized that I probably started out too basic with Anki. I didn't know what "cloze" type cards were, and they seem to be very useful for a few lists I want to memorize!


I find it very exciting that spaced repetition is find some interest here recently.

There recently was another discussion related to spaced repetition[0], and at the time I dismissed it. I mostly thought "huh, bad memory never really held me back in software engineering, so I don't really care", and that with the fact that my factual memory is probably the worst it has ever been.

Fast forward a month. My biochemistry university studies have started, and I'm absolutely in love with spaced repetition. It's been one of the best learning experiences of my life. I don't think I've ever acquired that much (factual) knowledge about a new field in such a short timeframe, though there a probably other factors at play too, like motivation.

I write Anki cards on my computer and sync them to my smartphones, and then do them on my commute and before I go to sleep. Every time I go through some of my course material and encounter something where I'm not sure what it means, I note the word (with context) down in my notebook and then later sit down to write a Anki card for it.

The first exams for the semester are still to come, but largely thanks to spaced repetition, I feel very confident about them. Really looking forward to applying it in other parts of my life in the future!

[0]: https://news.ycombinator.com/item?id=19529236


A very useful and quick reference to be productive at Anki itself is Anki Essentials and after working through that, I find my useful using Anki the right way. Though I don't think having multiple decks is a good idea and have found using single deck to be much more mimicking how consolidation of what you have learnt happens in real life, randomly across different subjects.

Does anyone here have tips to memorize the standard library of the programming language you use say Python using Anki? While yes there is the quick look up to the lib using a standard dev environment, what I am pointing to is having to lookup for details for the library function assuming that you know the function is the right choice because you have added it your long term memory. If this is useful for nothing, it will be useful at least for the algorithmic interviews which should be aptly named harakiri :-) [2]

[1] https://alexvermeer.com/anki-essentials/ [2] https://en.wikipedia.org/wiki/Seppuku


I’ve been using spaced repetition to do algorithm puzzles. I find them really fun, and I’ll be looking for a job in a little less than a year. So, I thought I’d get a jumpstart on preparing.

I don’t use them to memorize the answers, but more as a cue to do the problem. I add 1-5 new cards a day. The cards aren’t facts, but actual problems. It’s been really effective so far. I can pretty easily recognize what the pieces I need to apply on novel problems much more quickly.


Can you share an example card? I can’t picture what you’re doing.


Other examples of cards that I made and found useful during my job-hunting process: https://cards.c13u.com/browse[0]

[0] Disclaimer, I made cards.c13u.com for a class project


You copied the link incorrectly, it throws a 404. https://cards.c13u.com/browse works instead.


Probably you need to be logged in to see this


Yea, I just use the types of problems that you’d see on Leetcode or any of the other code practice sites.

The one that I did this morning was:

If given the root of a binary tree, calculate the number of unival subtrees contained in the tree.

The back is usually either blank, or has a space and time complexity that I should get if I did it correctly.

It’s important to make sure you solve it at least once, and understand the solution well before you make a card of it.


Are you using the ANKI method like what was suggested in the article?

Also, what kind of problems are you putting into the cards? Are there any "must-know" interview questions that we should all study and memorize?

Please share an example card.


I read the article a long time ago, so I don’t remember what you mean by the ANKI method. Anki is a program that assists you by automatically serving you cards. The article is quite long, but very good.

I’ve given 3 examples in some other replies, but I’d say that it’s most beneficial to keep some common stuff in the front of your mind. Like various operations on graphs and trees, how exactly to formulate dynamic programming subproblems, etc. For that I’d say it’s better to have a lot of different problems in the deck for those types of those things, rather than memorizing definitions of DP and linked lists.


Very cool! What format/content does a typical card contain for this category of algorithm cards you use?


It’s usually just a problem and sometimes a simple example I can write as a test case for correctness.

Some cards I’ve put in lately are:

Calculate the level of a binary tree with the minimal sum of its elements.

Reconstruct a binary tree given an inorder and preorder list of the nodes.

The backs are usually blank or space and time complexities that I should be able to achieve.


I did this exact method for my last round of algorithm interviewing and found it very helpful.

I also did this for various college courses I’ve taken, and I’ve found that to be tremendous for practicing and recognizing patterns.


I use Anki, but I have the problem that I tend to associate the specific form of the question, a particular latex formula etc. with the answer. As soon as I look at the question or read the first couple of words, the answer already pops up, so I'm afraid that I don't actually learn to remember the concepts but rather question/answer pairs. Does anybody have some recommendations for dealing with this problem?


Good recommendations so far, but personally I find one method to be the most effective : multiple question versions in conjunction with randomization

Say you have decided to brush up on your first grade math, and you make a card for multiplication. Instead of just making one "2 2 = 4" card, you make one card containing three potential questions, so you add "5 4 = 20" and "3 * 3 = 9" to the same card. Each time this card is scheduled to appear, Anki will randomly choose one of the questions to display.

This way you prevent your memory from overfitting to the irrelevant context and force it to remember the actual concept, because you have to use your knowledge of how multiplication works every time. You have to pick a large enough selection of possible questions for that to work, for me three is usually enough. Additional bonus : anki reviews get less tedious, because you don't feel like always seeing the same old stuff.

Here's a template for creating such cards: https://ankiweb.net/shared/info/947272864


So spaced repetition is of course designed for memorizing facts, not learning and integrating complicated concepts, but the danger of memorizing a particular visual presentation of the question prompt is well taken. My only suggestion would be to prepare multiple versions of the same question. This might initially lead to the different versions being collectively shown more often than the ideal spaced repetition frequency for a single fact, but assuming learning is transferring between version (i.e., they are reinforcing each other), maybe it works out in the long term?

Very interested to hear what others suggest.


Disclaimer: I've not been using Anki for too long

I had the same problem when I was initially only put in definitions into Anki. I was able to say the definition 1:1 but didn't form the concepts very well. My 2 (very similar) solutions for this (which I found to work pretty well), are:

1. Cross-referencing cards. I started editing a lot of cards, to add cross-references to other definitions that I also have cards for. For all the new cards I write now, I always try to have at least one reference to something that is on another card.

2. Meta-cards. I specifically add cards that test the knowledge of concepts, and me explaining it by adding cards like "Explain the difference between A and B".

3. Epic cards. When I started learning the periodic system of elements, along with the individual elements, I added a card for each group, a card for each group where I also have to know the element symbol, and a card for each group where I have to know the element symbol and atomic number. Obviously, I can only correctly solve a card if I know the easier versions of it, but it both provides a bigger goal to work towards, and also helps to put the smaller individual pieces of information I'm am learning into a context.


My general strategy around that is have the least amount of context possible. If context is necessary, then create multiple cards around it. For example, for a cloze deletion card, it's not uncommon I have up to 3+ clozes for a single note.


I have the same problem once in a while. Here are some things you can try.

1. Slow down your review session. I run into this problem mostly when I lose discipline and start skimming the question.

2. Put more effort into breaking info down into very atomic parts. For complex info, I'll still put in a high-level card, but I'll also put in cards that represent pieces of the info.

3. Add different cards for different methods of recall. I frequently add pairs of cards. For example, I added both "what is Betteridge's law?" and "what adage states that any headline that ends in a question mark can be answered with no?"


Oh! cool! As someone who's bad with remembering names (but who feels that using names is REALLY important to making people feel welcome) I use spaced repetition to better match names to faces as a Meetup co-organizer.

I basically wrote a CLI tool to auto-generate an Anki deck from a Meetup.com event URL, using the publicized names and profile pictures :) https://github.com/CivicTechTO/anki-meetup-memorizer

We have a script that runs weekly via CircleCI, before the event, and drops the file into a GDrive folder for me to easily load up on the way to the Meetup: https://github.com/CivicTechTO/civictechto-scripts/blob/b1cb...


Here's an obscure video[1] of Jeremy Howard (Fast.ai) talking about how he used spaced repetition to learn Chinese and which helped him to better other students learning Chinese in a university in China!

[1]: https://www.youtube.com/watch?v=HwH7bJibmdM


I love the layout of this website and how well it uses the horizontal space.

I wish more websites would create templates like that.


Not so much to complain, but a thing I noticed while scrolling fast is that the screen is black for a split second, and then text appears. I wonder why that is?


Probably because he’s using google fonts.


It doesn't use Google Fonts, and even if it did, I don't see why that would cause screen rendering issues on scroll (as opposed to rendering delays on initial page load).


what browser?


The screen went white for me in iOS/Safari.


I've had very good results using memrise's spaced-repetition tool to learn Korean vocab. I find it's a good way to get words into this unindexed "staging area" in my brain. And then when I need them during a real life situation, they sort of pop into my consciousness out of nowhere.

I also built a tool to build spaced repetition flash cards from the captions in YouTube videos. This uses the SuperMemo-2 algorithm, the same algorithm as Anki. This has helped me a lot with my listening comprehension.

https://www.captionpop.com/flash-cards/splash


A great book that covers this topic is Human Memory by Gabriel Radvansky. A used copy of the second edition goes for around $15.


Been using space repetition since 2013. It's unbelievably useful. Wrote about it in my blog with examples: https://www.blinkingcaret.com/2016/05/04/hack-your-brain-lea...


What kind of things should everyone memorize using spaced repetition?

Most people seem to use it to learn languages and technical subjects. I'm looking for things that most people would find useful in their daily lives:

- Social security number

- Credit card number

- Passport number

- Multiplication tables

- Unit conversion tables

- Emergency phone numbers

- Recovery codes and passwords

- Basic recipes and cocktails

- First aid techniques

- Survival techniques

- Phonetic alphabet

- NATO phonetic alphabet

- Basic words in foreign languages


This is a great essay. An exciting practical use case is seen in Duolingo. They use a regression model to learn the half-life of individual words for users.

[0]: https://www.youtube.com/watch?v=nvgGSg-VWAE


It is nice that this is reposted so often here. It is quite apposite to the nature of the phenomenon.


I have created a flashcards app for Android that tries to use sort of a spaced repetition system, just if anyone is interested in trying: https://play.google.com/store/apps/details?id=flashcards.wor...


If you are using Evernote, I've built this little tool that adds spaced repetition and flash cards capabilities to your notes & highlights. https://neuracache.com/


So, Vulcan teaching technique works?


[flagged]


I see you promoting your app/site constantly. I don't understand why this isn't against the rules on hn.


> I see you promoting your app/site constantly. I don't understand why this isn't against the rules on hn.

You can vote and you can flag. The source of the app in question is available on Github with a GPL V3 license.

Personally I'm happy to see someone so passionately tackle what I find to be an interesting problem. He's open and approachable. He should be encouraged. He's not advertising the monetized portion of the app (saving to Firebase, which he has to pay for himself.)

As he said, it's not "there" yet, but how else do you get there without becoming so intimately involved with a problem? I'm rooting for Polar and happy to see regular updates coming in through the channels I monitor.


It isn't against the rules to link to one's own work in contexts where it's relevant. It is against the rules to use HN exclusively for promotion. This falls somewhere in between, but https://hn.algolia.com/?query=getpolarized.io%20by:burtonato... is probably a bit excessive.


What I find interesting about that list is that it makes for a decent story as you read through his comments. It shows the struggles of building an open source app and attempting to make that work sustainable. You see the reason he started it, the technical challenges and the difficulties of raising funds. If someone else wanted to do the same, this is a great search string. I would be hesitant to try as he appears to be traveling a rough road.


I just saw that something that I am working on might be relevant to the topic. Same way that people post relevant links to the post.




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: