How to learn in batches with SRS?

[Edsko de Vries]

When starting to learn a new vocabulary list, I like to learn in batches. Study the first 20, re-test on the ones that I got wrong, then study the next 20, etc.

With the Random card selection system I can do this by setting "Max number of cards" to 20. But this setting is not available when using Repetition-spaced card selection. The closest corresponding option I can find is Limit new cards, which the manual describes as "a very useful option that lets you limit the pool of non-“learned” cards to a finite number; this keeps you from being overwhelmed by more new words than you can manage at a time, so if you add 8000 new flashcards to your database you won’t suddenly be expected to start memorizing all 8000 at the same time."

Indeed, that sounds good. But I can't quite make it work. Suppose I enabled Limit new cards by maximum unlearned with count set to 20. Then:

1. Starting with a fresh scorefile, I will get 20 cards in my first test. Suppose I get 15 right and 5 wrong. Since I enabled the Repeat incorrect cards I will be retested on those 5, and let's say I get them right now. So far this is perfect, this is what I was looking for.
2. After this first test Pleco now considers me to have 5 "Unlearned" cards -- best I can figure out, a card is considered "Unlearned" if it has been reviewed at least once but is not yet considered learned.
3. Now on my next test I only get 15 new cards. Suppose again I get 5 wrong. Now I have 10 "Unlearned" cards.
4. Next test: 10 cards; if I again get 5 wrong, there are now 15 "Unlearned" cards
5. Next test: 5 cards. If I get all of them wrong, there are now 20 "Unlearned" cards.

So far this is reasonable-ish. It's not quite batches of 20 because the "batch size" is reduced by the number of currently unlearned cards, but it's okay. But now I have a problem. Suppose I want to learn another batch. Since Pleco now considers me to have reached my maximum of unlearned cards, it will say "Session complete" when I try to start a new test. It does give me an option to continue regardless, but if I do, I will then be asked to review all cards I have reviewed before. In the example above, that is 20 + 15 + 10 + 5 = 50 cards. If during this review I will get some of the cards right that I got wrong in the previous session, they are no longer considered Unlearned (depending on the If review early setting) and so I can start to study some new cards again.

But this isn't very convenient. I don't want to review all new cards -- in fact, I'd rather not review any cards at this point, I just want to learn some new. Especially when I have been working with this vocabulary list for a while, reviewing all previously studied cards would be far far too many. The cards that I got right before I'd like to review when they are up for review, depending on the regular SRS settings.

Moreover, the same problem arises a few days later when the cards that I did study are up for review. Now when I start a test I will also get all cards that are up for review, rather than study them in batches. (This is somewhat less of an issue because when reviewing previously learned cards it's not quite so important to focus on smaller groups at a time, but I think it's the same problem from a technical point of view.)

Am I using the SRS system in the wrong way? For now I can see two workarounds. The first workaround (the one I am using now) is to use Simple card selection for learning new cards (with a card filter that removes any cards with score higher than some threshold), and use SRS only to review (using the same scorefile). The other workaround is to split the vocabulary list into smaller segments; although Pleco makes this splitting easy enough, it's still somewhat of a pain when studying when the segments are as small as blocks of 20, say.

Any suggestions?

 

[mikelove]

There isn't currently a way to do this totally automatically unless you want to limit new cards to a certain number per day instead of a certain number altogether - that's the only thing that gives you SRS batches independent of your unlearned card count. We probably are going to add an "add X new cards to your pool" option in the "session complete" popup in a future update but I can't say exactly when that'll happen.

However, it seems like you could get closer to the behavior you want by adjusting Pleco's settings so that incorrect cards a) come up again very quickly and b) can be promoted to 'learned' if on one of those later reviews you answer them incorrectly. For a), set "day type" in Card Selection to "calendar" and set "minimum score" in Scoring / Tweak Parameters to 99; that will cause incorrect cards to come up again on every test until you answer them correctly. For b), turn off "only change 1x per day" in Scoring and set the "card is learned if" criteria in Card Selection to "correct total" or "correct in row" with amount '1'.

 

[Edsko de Vries]

Thank you for your quick response. It's still not quite what I was hoping for though. I know it's hard to please all users (I'm a software developer myself but I'm just surprised that other people don't have the same problem. For instance, I now have a category with some 500 words, all of which I have studied before in the past and I just want to review. But if I use SRS, even with your suggestions, and start a test, I will then get a test that has all 500 words in them. If I don't know a word correctly, it won't come up again until I have answered all other 499 words, right? If this was an actual physical deck of cards sitting here on my desk, I'd take a batch of 'em, make sure I know them, and then move on to the next batch. Kind of surprised other people are not bothered by this, I wonder how they do it? I must be using this in a non-standard way, but I'm not sure how

It seems to me that the only thing that is required is to enable the "Max number of cards" setting that is available in Random card selection also in Repetition Spaced. I realize the manual explicitly states that "This option is hidden in repetition-spaced tests, since in order for those to work correctly you need to study all of the cards that are currently due to be tested, though you can exit a repetition-spaced test early (or save it to resume later) if you run out of time." but I don't quite see why this is a problem.

 

[mikelove]

To be honest, reviewing all 500 words in a category isn't really the sort of thing that you'd want to do with SRS - the whole idea of SRS is that it's showing you words at specific times in order to minimize the number of times you have to repeat each word and therefore maximize your learning efficiency. So if our SRS is trying to make you review all 500 words in that category then that might mean that your settings are adjusted to repeat cards too often - have you changed any of the options in Tweak Parameters besides the ones I mentioned?

For what you're talking about, I think it would probably make more sense to do a separate, non-SRS test (can keep its settings ready at hand via Profile) with scores not recorded ("scoring system" in Scoring set to "none") and the "repeat incorrect at end of test" option in Commands turned on; that will let you review however many cards you want at a time, and make you repeat incorrect cards at the end of the test until you get them right.

The reason we don't let you set a specific number of cards in SRS (and it's something people have been asking for ever since this iteration of our flashcard system first debuted in 2008) is because SRS only works if you keep up to date with your reviews; if you have 200 cards due to review today and you only review 100 of them, tomorrow you might have 300 cards due to review, and the 100 of those that you missed the previous day are more likely to be forgotten + need to be learned again. So giving people a way to review only some of the SRS cards that they have due would effectively be rendering SRS useless for them.

If you interrupt an SRS test in the middle of reviewing (with the [X] button), it will still offer to repeat incorrect cards (if you've enabled that option in Commands), and it will carry on from where you left off (not repeating the other cards you studied today) when you start the test again. So you can review SRS in batches that way, but it'll still be obvious that you have X number of cards left to get through if you want to keep apace with your reviews.

 

[Edsko de Vries]

No, I didn't change any settings. I guess the problem is that these are word lists that I revised in one setting (I knew most of them) and therefore now that the 4 days are gone by, every single one of them is up for review again. So now I'm going through all 500 again, and since I'm getting the majority right but some wrong, I guess that means that over time the number of cards that Pleco asks me in an SRS setting will become more manageable.

So, yes, I think I shall stick with my original workaround, the one you recommend now too -- use a Random profile for learning and a Spaced Repetition profile for revising.

非常感谢你的帮助！Overall Pleco is awesome, I use it every single day

 

[mikelove]

Ah, that explains it. You could help the problem a bit through a slightly hack-ish trick in Batch Commands: go into Organize Cards, navigate to your category, tap Edit, Batch, and under "Score" run a "change score" command to randomly increase scores by up to (say) 300 points; that will randomly adjust cards' intervals to spread them out over a few days instead of having them all come up the same day.

(we're working on a way to automate this - there should be a prompt saying "you have a lot of cards due today, would you like to study them over the next few days?" - but that's harder than it sounds because we also have to ensure that you don't have an even larger number of cards coming up tomorrow / aren't about to get into a slippery slope of reviews piling up, in which case we'd need to help in a more dramatic way)

 

[HW60]

The reason we don't let you set a specific number of cards in SRS (and it's something people have been asking for ever since this iteration of our flashcard system first debuted in 2008) is because SRS only works if you keep up to date with your reviews; if you have 200 cards due to review today and you only review 100 of them, tomorrow you might have 300 cards due to review, and the 100 of those that you missed the previous day are more likely to be forgotten + need to be learned again. So giving people a way to review only some of the SRS cards that they have due would effectively be rendering SRS useless for them.

The problem of SRS is that sooner or later the day comes when you do not have the time to repeat the due amount of flashcards. If that happens for 3 weeks in a row, you have a real problem: lots of cards to review, and no chance to catch up. Therefore you do not need to "give people a way to review only some of the SRS cards" - it comes all by itself. But what you could do - and we both remember lots of discussions about it - is to give people a reasonable way to catch up with a reduced number of cards. You have to set goals which you want to achieve with such a system, e.g. repeat cards with a low score first because you otherwise will forget them completely (Gato's original idea), learn new words, and more.

I followed an idea very similar to Gato's for more than a year now and am very satisfied with the results. I set the upper score limit to a value with which I get the desired number of cards. This upper score limit steadily increases, and I repeat all the problem cards with a low score until they get a higher score. Reviewing of cards with a higher score is postponed until the upper score limit slowly gets higher.

For new cards and for speeding up the process of catching up I made a copy of my scorefile and do standard SRS without an upper score limit for new words and for a slowly increasing amount of separate categories. You could invent some statistical figures to show the result of the catch up process: number of overdue cards in % of all cards, development of the upper score limit etc. I do it with Excel, but I would prefer to spend the time required for that with reviewing flashcards.

What is most important for me: I do decide how much time I spend repeating flashcards, not a program, even if it is as outstanding as Pleco is. Pleco could support this process, which is not too easy otherwise, but this is not a new idea.

 

[Shun]

I would be perfectly happy with a behavior identical to that of a physical stack of cards, and better statistics and logging. (with card filtering like today) I think Mike has said in this forum that he has something great up his sleeve that will gradually be introduced in the coming releases. He's already added the Repeat-within-session option that I welcome very much. I'll gladly wait for that day.

 

[Edsko de Vries]

I now use random selection to study new words, and SRS for revision. I wrote it up as a blog post in case it's helpful to anyone: http://www.edsko.net/2015/01/16/study-with-pleco/.

 

[mikelove]

This is great, thanks for posting it! (much better than my own explanation of scorefiles, frankly )

 

[sobriaebritas]

I know it's hard to please all users (I'm a software developer myself but I'm just surprised that other people don't have the same problem. For instance, I now have a category with some 500 words, all of which I have studied before in the past and I just want to review. But if I use SRS, even with your suggestions, and start a test, I will then get a test that has all 500 words in them. If I don't know a word correctly, it won't come up again until I have answered all other 499 words, right? If this was an actual physical deck of cards sitting here on my desk, I'd take a batch of 'em, make sure I know them, and then move on to the next batch. Kind of surprised other people are not bothered by this, I wonder how they do it? I must be using this in a non-standard way, but I'm not sure how .

Hello, Edsko:
Thanks to you I've realized that I have the same problem, but was not able to explain it to so clearly as you.
I've just read your linked blog post. IMHO, it's a nice, clear and systematic combination of insights into learning Chinese as a foreign language acquisition and making the most of Pleco's present flashcard system.
But I'm not sure about one thing, though: for instance, before beginning a test session of “Listening (Review)”, which card category do you choose in Basic Settings? I suppose you choose the "category with some 500 words, all of which I have studied before in the past and I just want to review". Is that so?
Thanks again for sharing you approach, Edsko.

 

[Edsko de Vries]

@sobriaebritas I'm glad you liked the blog post. As for reviewing large categories -- you are right that I did not address this explicitly, but with the way I've set things up now it bothers me less. It's true that when I review a large category that I haven't reviewed in a while I might still get a test with a lot of words. But I just review as many as I have time for; you can stop the test whenever you want. And here's the key thing: during revision I don't worry so much about the words I get wrong. Those words will re-appear in my "Listening (New Words)" test, where I study them like any other word I don't know (get right three times in a row, study in small batches, repeat incorrect words after each batch).

 

[MrGreen]

@sobriaebritas @sobriaebritas I'm glad you liked the blog post. As for reviewing large categories -- you are right that I did not address this explicitly, but with the way I've set things up now it bothers me less. It's true that when I review a large category that I haven't reviewed in a while I might still get a test with a lot of words. But I just review as many as I have time for; you can stop the test whenever you want. And here's the key thing: during revision I don't worry so much about the words I get wrong. Those words will re-appear in my "Listening (New Words)" test, where I study them like any other word I don't know (get right three times in a row, study in small batches, repeat incorrect words after each batch).

Great blog post edsko. I'm going to give your system a try. Hopefully it'll help me learn Chinese faster.

 

[pdwalker]

Edsko,

What about score files? Do you create one score profile per test profile, or do you have it set another way?

 

[Edsko de Vries]

@pdwalker, yes. This is kind of important; a high score in one direction does not imply you also know the word in the other direction. The blog post details my approach:

So, I suggest practicing each of these 4 directions separately, and thus set up 8 profiles: for each direction one for revision and one for learning new words. Note that you will want to use the same scorefile for learning new words and for revision, but a different scorefile for each direction (so you will have 8 profiles, but only 4 scorefiles).​

 

[pdwalker]

Hi Edsko,

Thanks for the quick reply. I think I almost get it now.

Another question: for each "direction" you create two test profiles. "new words" and "review". How do you manage your day to day learning?

Let's say I want to work on a list of 40 new words. Do I keep running the "new words" tests until the test list comes up empty (or I get bored), and from there on use the "review" tests?

And as I add new words, I return to the "new words" tests until I've "learned" them and continue with the "review" tests?

As an example, how many "review" vs "new words" tests would you do per day as you are learning?

My understanding is, I add new words. Run the "new words" tests until I have learned them, then daily run the "review" tests until there is nothing left to review for that day. Does that sound right?

 

[Edsko de Vries]

My daily ritual looks like this: I first run the "review" test and review all the words that Pleco tells me are up for review that day. For me this tends to be an average of about 150 words or so, sometimes less, sometimes significantly more. I run review first, because any words that I get wrong will then re-appear in the "new words" test. After that I run the "new words" test. And like you say, you learn just as many as you have time for / until you get bored.

As for how many new words I learn per day, it very much varies, and I partly let Pleco dictate this as well. If Pleco gives me 900 words to revise (it's been known to happen..) then I'll probably not learn any new words that day (if I even make it through all words to be revised..). Also, if my "new words" primarily consists of words that I had learned before but got wrong in revision, I actually don't use the limit of 20 in the "new words" test. In that case, I just start the test on all "new" words, and I just stop the test when I've gotten a certain number wrong (say, 10 of them). At that point Pleco will ask you the words you got wrong, so it basically allows you to vary that limit of 20 depending on the difficulty of the words. Actually, the same principle applies when I'm learning new words lists. Sometimes even a set of new words may be relatively easy; for instance, as my vocabulary grows it becomes somewhat easier to learn new words because I already know the individual characters that make up the word; so I might still not impose the limit of 20 and just enable all words and stop the test when I got too many wrong.

But yes, when you are really learning a brand new set of words, just learn a couple of batches, until you get bored (I think there's also some kind of law of diminishing returns here; for me anyway there just isn't much point learning more than a certain number because I'll have forgotten them in the morning). I would however recommend revise first, and only then learn new words.

 

[pdwalker]

Thanks Edsko,

I've gone and setup 10 testing profiles using 5 score files, and I run through them in this order:

1. Listen, review
2. Listen, new words
3. Reading, review
4. Reading, new words
5. Speaking, review
6. Speaking, new words
7. Writing, review
8. Writing, new words
9. Pinyin, review
10. Pinyin, new words

The "new words" I'll run until I'm bored, or I've finished the list for the day - it depends on how many new words I've added to my lists.

The "review" settings, I'll complete daily, at least so far.

It seems to work for me.

I've put together the exact settings I use for each of the testing profiles into a spreadsheet, which I've attached in case anyone else is interested. These settings are based on <a href="http://www.edsko.net/2015/01/16/study-with-pleco/">your blog post</a>

 

[pdwalker]

oops. Some minor mistakes in the attachments. Revised.

 

[pdwalker]

For those interested in this kind of stuff, I've updated my test settings based on what I've learned while using the pleco flash cards. The changes are marked in red.

Basically, for listening tests, I've found some of the audio samples to be unclear, so I display the pinyin to help make sure I am hearing it correctly.

For reading new words, I've changed the test to multiple choice.

For pinyin new words, I show the character, definition and audio.